-- Logs begin at Fri 2022-06-24 07:52:01 UTC, end at Fri 2022-06-24 09:06:23 UTC. -- Jun 24 08:27:36.481510 dv-cinder-845178-9-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 24 08:27:36.489840 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: *** Starting uWSGI 2.0.18-debian (64bit) on [Fri Jun 24 08:27:36 2022] *** Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: nodename: dv-cinder-845178-9-iscsi Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: machine: x86_64 Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: clock source: unix Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: pcre jit disabled Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: detected number of CPU cores: 4 Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: current working directory: / Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: detected binary path: /usr/bin/uwsgi-core Jun 24 08:27:36.491880 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: your processes number limit is 47680 Jun 24 08:27:36.492585 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: your memory page size is 4096 bytes Jun 24 08:27:36.492585 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: detected max file descriptor number: 2048 Jun 24 08:27:36.492585 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: lock engine: pthread robust mutexes Jun 24 08:27:36.492585 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: thunder lock: enabled Jun 24 08:27:36.492585 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 24 08:27:36.492585 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 24 08:27:36.536284 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: Python main interpreter initialized at 0x55b61a02e9c0 Jun 24 08:27:36.536284 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: python threads support enabled Jun 24 08:27:36.536284 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: your server socket listen backlog is limited to 100 connections Jun 24 08:27:36.536284 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: your mercy for graceful operations on workers is 90 seconds Jun 24 08:27:36.536595 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: mapped 403077 bytes (393 KB) for 2 cores Jun 24 08:27:36.536694 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: *** Operational MODE: preforking *** Jun 24 08:27:36.536960 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: *** uWSGI is running in multiple interpreter mode *** Jun 24 08:27:36.536960 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: spawned uWSGI master process (pid: 80397) Jun 24 08:27:36.537150 dv-cinder-845178-9-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 24 08:27:36.539432 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: spawned uWSGI worker 1 (pid: 80406, cores: 1) Jun 24 08:27:36.539432 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: spawned uWSGI worker 2 (pid: 80407, cores: 1) Jun 24 08:27:36.539432 dv-cinder-845178-9-iscsi devstack@keystone.service[80397]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 24 08:27:41.963975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:27:41.963975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: from cryptography.utils import int_from_bytes Jun 24 08:27:41.964776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:27:41.964776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: from cryptography.utils import int_from_bytes Jun 24 08:27:42.009287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:27:42.009287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: from cryptography.utils import int_from_bytes Jun 24 08:27:42.009836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 24 08:27:42.009836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: from cryptography.utils import int_from_bytes Jun 24 08:27:42.013287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 08:27:42.013484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 08:27:42.013679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] command line args: [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 08:27:42.013951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 08:27:42.014133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ================================================================================ {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 08:27:42.014388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] admin_token = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.014635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] config_dir = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.014824 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.015005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] config_source = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.015186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] debug = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.015414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.016495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.016495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.016495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.016495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.016749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.016871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_config_append = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.017008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.017186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_dir = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.017367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_file = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.017601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.017851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.018101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.018348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.018527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.018704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.018951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.019233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.019481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.019714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.019951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.020186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.020432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.020677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.020959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.021436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.021642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] publish_errors = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.021850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.022077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.022251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.022498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.022732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.022967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] standard_threads = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.023474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.023721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.023897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.024072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] use_journal = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.024248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] use_json = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.024425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] use_stderr = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.024601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] use_syslog = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.024832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.025088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.025327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.025588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.025826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.026076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.026313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.026560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.026796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.027044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.027315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.027569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.027803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.028046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.028283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.028529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.028773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.029009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.029264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.029516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.029760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.030002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.030253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.030513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.030765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.031012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.031333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.031570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.031802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.032038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.032286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.032534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.032765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.032995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.033241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.033486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.033723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.033972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.034205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.034440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.035185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.035441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.035674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.035907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.036139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.036377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.036624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.036857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.037094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.037344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.037579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.037815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.038062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.038292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.038529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.038766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.039002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.039261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.039496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.039728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.039972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.040208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.040449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.040681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.040938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.041173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.041404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.041640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.041874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.042141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.042378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.042626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.042871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.043107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.043648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.043886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.044134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.044372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.044610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.044857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.045096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.045336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.045574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.045812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.046050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.046312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.046553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.046795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.047033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.047302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.047540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.047777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.048012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.048243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.048473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.048703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.048937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.049167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.049407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.049642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.049890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.050126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.050910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.051180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.051447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.051678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.external = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.051908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.052201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.052433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.052665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.password = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.052896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] auth.token = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.053134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.053368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.053612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.053841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.054086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.054332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.054562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.054798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.055041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.055326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.055573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.055811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.056053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.056288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.056495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 24 08:27:42.056632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.056765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 24 08:27:42.056893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.057030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] command line args: [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 24 08:27:42.057235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.057370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 24 08:27:42.057496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.057621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ================================================================================ {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 24 08:27:42.057747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.057874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] admin_token = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.058010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.058137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] config_dir = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.058137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.058373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.058504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] config_source = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.058630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.058782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] debug = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.058782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.059055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.059183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.059356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.059496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.059628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.059755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.059880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.060007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.060138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.060276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.060664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.060798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_config_append = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.060798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.061037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.061164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_dir = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.061298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.061426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_file = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.061582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.061718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.061845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.061972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.062110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.062238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.062363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.062506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.062506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.062757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.062885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.063014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.063142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.063303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.063443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.063572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.063707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.063852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.063991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.064128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.064257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.064383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.064520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.064658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.064785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.064916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.065044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.065176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.065315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.065446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.065446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.065685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.43/identity {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.065823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.065980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] publish_errors = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.066113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.066240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.066368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.066495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.066495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.066743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.066869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.066995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.067122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.067275 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.067423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] standard_threads = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.067551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.067692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.067825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.067953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.068079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.068214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.068342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.068470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] use_journal = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.068470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] use_json = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.068714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.068842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] use_stderr = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.068977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.069105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] use_syslog = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.069231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.069368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 24 08:27:42.069502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.069629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.069764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.069892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.070650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.071368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.071523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.071653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.071780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.071907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.072810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.073837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.074878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.075973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.076885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077275 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.077914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.078934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.079901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.080951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.081851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.082899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.083983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.084889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.085959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.086950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.087921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] role.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] role.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.088961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.089874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.090915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.091876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.092904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.093968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.094876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.095977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.096977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.097111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.097736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.097873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.external = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.098928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.password = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] auth.token = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.099999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.100914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.101944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.102985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.103128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.103337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.103517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.connection = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.103670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.103820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.103961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.104920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.105876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.106968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.107947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80406) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.108909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.110096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.111187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.112342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.114078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.115174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.116329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.117719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.118096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.118229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.118410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.118637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.118881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.119116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.119373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.119608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.119842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.120084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.120333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.120568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.120818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.121046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.121288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.121521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.121753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.121987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.122228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.122465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.122694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.122927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.123163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.123434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.123678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.123911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.124156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.124404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.124636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.124878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.125111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.125345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.125576 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] role.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.125818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] role.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.126044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.126289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.126533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.126777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.127019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.127290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.127532 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.127774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.128016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.128246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.128512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.128757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.129001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.129251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.129652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.129888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.130135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.130388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.130635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.130881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.131115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.131391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.131643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.131878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.132108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.132338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.132420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.132568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.132649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.132795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.132884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.133055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.133301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.133538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.133779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.134008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.134242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.134477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.134720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.134954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.135224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.135472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.135707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.135948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.136187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.136431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.136663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.136897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.137128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.137370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.137616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.137844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.138083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.138374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.138631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.connection = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.138872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.139115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.139373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.139613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.139842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.140078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.140317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.140550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.140793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.141026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.141267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.141346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.141500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.141745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.141980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.142222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.142450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.142688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.142920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 24 08:27:42.143089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80407) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 24 08:27:42.169451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.169679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.169897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.170007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.170188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.179069 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.197455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.197687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.312509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.323814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.362124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.362363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.364101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.364334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.427694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.427917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.428984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.429202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.452194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.452434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.452773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.452877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.452957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453412 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.453801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.464564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.464726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.477913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80407) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.478685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80406) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 24 08:27:42.496032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.496239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:27:42.496573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.496655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.496772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:27:42.496869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:27:42.497038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.497121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.497294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.497378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 24 08:27:42.497553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.497654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.497794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.497875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.498063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.498321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.509454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.509729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.509811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.509996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.510078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.510270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.510351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.510551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.518410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.518851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.521719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.521998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.522184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.522391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.522473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.522673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.522763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.522950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.523032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.523500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.523589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.523768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.523852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.524909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.525097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.551597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.552036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.553192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.553467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.553663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.553943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.558700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.558976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.559203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.559297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.559618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.559904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.567110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.567369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.567761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.568046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.571957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.572239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.572518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.572598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.572798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.572891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.573957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.574143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.574411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.592499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.592774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.593199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.593478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.597378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.598162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.601568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.601823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.601904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.602218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.606843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.607288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.610455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.610903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.614814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.615111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.615194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.615440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.615550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.615732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.615840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.616009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.616092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.616286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.616377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.616568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.635690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.636063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.636145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.636332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.636414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.636601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.636685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.636884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.644505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.644832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.646222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.646494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.646610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.646796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.649831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.650105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.650271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.650386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.650550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.650661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.650832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.650942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.651120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.651441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.664398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.664677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.664811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.664963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.665090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.665242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.665376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.665520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.665660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.665793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.666067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.667167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.667485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.667782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.685565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.686121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.688861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.689455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.692774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.693365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.694368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.695107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.696030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.696307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.696587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.696774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.697052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.697349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.703995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.704874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.705606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.706482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.708861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.709147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.709427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.709697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.709777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.710072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.710465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.710749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.718875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.719159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.719471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.719881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.720167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.720455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.727859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80407) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.728138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.728415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.728688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.728868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80406) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 24 08:27:42.728952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.729971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.730919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.731044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.731192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.731327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80407) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.731496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.731771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.732043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.732310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.732588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80406) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 24 08:27:42.769438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b61a02e9c0 pid: 80407 (default app) Jun 24 08:27:42.770662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b61a02e9c0 pid: 80406 (default app) Jun 24 08:27:42.772750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcef9c-dad5-4807-a9aa-db61b36c8b9a None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:42.773044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcef9c-dad5-4807-a9aa-db61b36c8b9a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:42.773586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcef9c-dad5-4807-a9aa-db61b36c8b9a None None] PATH_INFO: `/v3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:42.777749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1/1] 10.222.0.43 () {50 vars in 791 bytes} [Fri Jun 24 08:27:42 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:50.460732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.461278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.461278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.465521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1/2] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:50.471240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:50.471679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:50.471798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:50.540060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: INFO dbcounter [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] Registered counter for database keystone Jun 24 08:27:50.560667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG oslo_db.sqlalchemy.engines [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80407) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 08:27:50.567310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writer thread running {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 08:27:50.743314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG passlib.handlers.bcrypt [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80407) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 08:27:50.743651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG passlib.handlers.bcrypt [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80407) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 08:27:50.797465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:50.799040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:50.921625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 2/3] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 451 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 08:27:51.056707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.126437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: INFO dbcounter [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] Registered counter for database keystone Jun 24 08:27:51.145131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG oslo_db.sqlalchemy.engines [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80406) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 24 08:27:51.151489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writer thread running {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 24 08:27:51.270937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b31528-56a0-4d14-87bf-8edc68762238 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.275451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.276127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.276413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.276692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.285318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.337358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING keystone.common.rbac_enforcer.enforcer [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 08:27:51.337646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57078938-d3ca-44a2-821e-e6fdd37503ac admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.347645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 2/4] 10.222.0.43 () {58 vars in 1324 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:27:51.353491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:51.367759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119838a8-5a1e-47c1-8372-10b80dc22889 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:51.372087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:51.373196 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:51.373484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:51.373758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] PATH_INFO: `/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:51.377778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] RBAC: Authorizing `identity:get_project(project_id=ffa0e0eb99bd42cebd2bc5b9d635fa5d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:51.428511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING keystone.common.rbac_enforcer.enforcer [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 24 08:27:51.428851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0f8098-7de4-4229-bbc3-51d5b9b6658c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:51.433311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 3/5] 10.222.0.43 () {56 vars in 1207 bytes} [Fri Jun 24 08:27:51 2022] GET /identity/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:53.417179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f923d-fac0-4a26-b2aa-f8c97ab78cc5 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.417679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f923d-fac0-4a26-b2aa-f8c97ab78cc5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.417781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f923d-fac0-4a26-b2aa-f8c97ab78cc5 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.421832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 3/6] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:53.427151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.427612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.427748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.446643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:53.448128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:53.456014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 4/7] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 24 08:27:53.478320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f923d-fac0-4a26-b2aa-f8c97ab78cc5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.490166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f923d-fac0-4a26-b2aa-f8c97ab78cc5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.494303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c4f7c-f047-4027-aae8-25a06dc1accb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.494927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c4f7c-f047-4027-aae8-25a06dc1accb admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.495227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c4f7c-f047-4027-aae8-25a06dc1accb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.495523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c4f7c-f047-4027-aae8-25a06dc1accb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.503160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c4f7c-f047-4027-aae8-25a06dc1accb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.505473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c4f7c-f047-4027-aae8-25a06dc1accb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.514176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 4/8] 10.222.0.43 () {58 vars in 1324 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:27:53.519929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:53.528993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07339b5c-b463-4d45-b8b3-6281b949f288 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:53.533316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbacf747-4038-468f-aa23-591cd0403ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:53.534167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacf747-4038-468f-aa23-591cd0403ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:53.534460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacf747-4038-468f-aa23-591cd0403ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:53.534733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbacf747-4038-468f-aa23-591cd0403ca7 admin admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:53.536100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacf747-4038-468f-aa23-591cd0403ca7 admin admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:53.537920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbacf747-4038-468f-aa23-591cd0403ca7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:53.539504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 5/9] 10.222.0.43 () {56 vars in 1198 bytes} [Fri Jun 24 08:27:53 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:55.605751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a9c0a-f577-441c-9224-29fbc4455801 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.606262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a9c0a-f577-441c-9224-29fbc4455801 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.606344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469a9c0a-f577-441c-9224-29fbc4455801 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.607885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 5/10] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:55.612663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.613597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.613994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.630267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:55.631522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:55.632993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fb724-1eab-4a1f-bd87-69e2b35f79e6 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.633297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fb724-1eab-4a1f-bd87-69e2b35f79e6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.633573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13fb724-1eab-4a1f-bd87-69e2b35f79e6 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.634907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 6/11] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:27:55.639829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.640123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.640407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.680267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG passlib.handlers.bcrypt [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80406) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 24 08:27:55.680679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG passlib.handlers.bcrypt [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80406) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 24 08:27:55.710614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 6/12] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:55.725839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:55.727453 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:55.733711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.756592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.825987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 7/13] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:27:55.834882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf0e57-8eb5-4d20-9dd1-064b44e83cf8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.848893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.868212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.869003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.869287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.869563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:55.880805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:55.891803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:55.892763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:27:55.893608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 7/14] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:55.899546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:55.950607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828cf3e-7f28-4cdc-bc0a-13ed04a40222 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:55.952900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.986561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:55.988905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:55.989836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:55.990124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:55.990402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.005116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b9376-153f-4a06-aa3f-068fc8a8ae0e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.039394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.040054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.040368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.040649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.041399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.041707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.043101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.054417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 8/15] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:56.058139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.059725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.059842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.155542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 8/16] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:27:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:27:56.157111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12707-649d-4871-bd85-ba6cb7ca6408 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.161734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.164535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.195907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.196788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.197076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.197358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.198576 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.200087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.201709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 9/17] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:56.207629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.221491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.262249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41700a76-a088-48b2-882b-b70de2fc8658 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.294897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.295707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.295991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.296263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.304282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f14f19-d72d-4617-864b-3e08793a630c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.315443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.317561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:56.338425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:56.339405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:56.340224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:56.340224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:56.340224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:56.340224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:27:56.340439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.340522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 9/18] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:56.340587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.340587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.340587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.341556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.343010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.344631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 10/19] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:56.345364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.350012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:56.403810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.429197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:56.447997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20196860-aafe-4848-bc5e-e888c17721a9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.448311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0809867b-5159-4747-8c07-cc6a9a1eeaf0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:56.481118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.481759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:56.481884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.482055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.482337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.482424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:56.482683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:56.482965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:56.483054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.483403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.483839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.domain.id=default, user.id=ca1ff62a97334431beecc4658e2e8e61) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:56.484157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:56.484802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d0abc-f8de-48db-bac6-f75030bfbb38 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.485598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c228b-48cc-4b3f-a434-323f0fce0403 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:56.492284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 11/20] 10.222.0.43 () {56 vars in 1322 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/role_assignments?user.id=ca1ff62a97334431beecc4658e2e8e61&scope.domain.id=default&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:56.502440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 10/21] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:27:58.796369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33da97-6299-4eed-9767-29b0d095ec58 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.796884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33da97-6299-4eed-9767-29b0d095ec58 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.796985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33da97-6299-4eed-9767-29b0d095ec58 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.798328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 12/22] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:58.804391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b1b57-72cb-4909-a2f1-194472e4a016 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.804822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b1b57-72cb-4909-a2f1-194472e4a016 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.805014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b1b57-72cb-4909-a2f1-194472e4a016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.828195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-625b1b57-72cb-4909-a2f1-194472e4a016 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:58.829724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-625b1b57-72cb-4909-a2f1-194472e4a016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:58.910442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 11/23] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:58.931000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b0d1-0053-443d-a0ed-8154f2505d49 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.931321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b0d1-0053-443d-a0ed-8154f2505d49 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.931602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38b0d1-0053-443d-a0ed-8154f2505d49 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.932986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 13/24] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:58.938612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:58.939147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:58.939697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:58.953659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b0d1-0053-443d-a0ed-8154f2505d49 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:58.962686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:58.964345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:59.031586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.062081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 12/25] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:59.112638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.311450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589c41a-b8ff-413b-824f-12c1d7d3d04f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.352613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.353767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.354254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.354742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.363916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38b0d1-0053-443d-a0ed-8154f2505d49 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.366264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.368022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-13ebcf7c-30b2-45cc-ab02-29904980ab71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:27:59.388315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:59.389197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:59.389997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:27:59.389997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:27:59.389997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:27:59.389997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:27:59.392337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 13/26] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:27:59.394782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70d637-81a8-488d-89a2-1aa45e710a51 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.395325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70d637-81a8-488d-89a2-1aa45e710a51 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.395800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70d637-81a8-488d-89a2-1aa45e710a51 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.398012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 14/27] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:59.399294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70d637-81a8-488d-89a2-1aa45e710a51 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.431573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.461812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a505-a9f1-471d-af3b-106a59d4e961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.461812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a505-a9f1-471d-af3b-106a59d4e961 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.461812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a505-a9f1-471d-af3b-106a59d4e961 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.461812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a505-a9f1-471d-af3b-106a59d4e961 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.483745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a505-a9f1-471d-af3b-106a59d4e961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b967a505-a9f1-471d-af3b-106a59d4e961 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:27:59.502515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:27:59.504038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:27:59.504792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:27:59.509336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 14/28] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:27:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 556 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:27:59.510809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.511098 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.515793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.536546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.558939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:59.560870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:27:59.600336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70d637-81a8-488d-89a2-1aa45e710a51 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.666268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.668782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.670050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.670504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.672581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.673191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.677917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4120ae-5bfe-4be6-ac91-1eec4d4c6997 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.692397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 15/29] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:27:59.693992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.697619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 15/30] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.699237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ba3b-a6a7-4a95-8e1e-8ac73e93c34f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.703586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ba3b-a6a7-4a95-8e1e-8ac73e93c34f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.704413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5ba3b-a6a7-4a95-8e1e-8ac73e93c34f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.708568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 16/31] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:27:59 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:27:59.716134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ba3b-a6a7-4a95-8e1e-8ac73e93c34f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:27:59.858145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.859153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece193e3-76e9-45c2-8e44-533be1607e37 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.874058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d5ba3b-a6a7-4a95-8e1e-8ac73e93c34f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:27:59.897310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.898213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.898892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.899339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.900267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:27:59.900739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.902319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f1fcde-1876-4fd8-a984-28199942c5ba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.904035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:27:59.912274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:27:59.913029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.913399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.913654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 16/32] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:27:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:27:59.913752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.914871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:27:59.915464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:27:59.915998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:27:59.916498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:27:59.916598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:27:59.934741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:27:59.936405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:00.021860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 17/33] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:27:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:00.021860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.075646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 17/34] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:27:59 2022] POST /identity/v3/projects => generated 301 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:00.076918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.189705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7f446-8c58-40e2-b656-60efd024dae3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.254486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.280232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da672e60-cd44-4225-937b-3668fd399a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.280920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672e60-cd44-4225-937b-3668fd399a37 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.281205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672e60-cd44-4225-937b-3668fd399a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.281483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672e60-cd44-4225-937b-3668fd399a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.282239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da672e60-cd44-4225-937b-3668fd399a37 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.283681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da672e60-cd44-4225-937b-3668fd399a37 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.289882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01135d3f-8418-41dd-8242-3be3b96676cf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.333889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.334774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.335058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.335487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.337159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.338719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.340614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 18/35] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:00.341303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.353751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.416269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 18/36] 10.222.0.43 () {60 vars in 1131 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/roles => generated 219 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:00.423341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2b30b-082c-446e-950c-c7b02895ccb3 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.423341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2b30b-082c-446e-950c-c7b02895ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.423341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e2b30b-082c-446e-950c-c7b02895ccb3 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.423341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 19/37] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:00.423341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2b30b-082c-446e-950c-c7b02895ccb3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.461756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244a1717-8c51-4388-b194-0ca8148dc68e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.505536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecf94d4-2e0c-496c-bece-cb3ecbcfaa95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.506224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf94d4-2e0c-496c-bece-cb3ecbcfaa95 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.506512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf94d4-2e0c-496c-bece-cb3ecbcfaa95 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.506793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecf94d4-2e0c-496c-bece-cb3ecbcfaa95 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.507623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf94d4-2e0c-496c-bece-cb3ecbcfaa95 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.509046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecf94d4-2e0c-496c-bece-cb3ecbcfaa95 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.526201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 19/38] 10.222.0.43 () {60 vars in 1131 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/roles => generated 225 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:00.527742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fd30-e2c7-4020-9a09-f02739fc7aeb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.528038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fd30-e2c7-4020-9a09-f02739fc7aeb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.528322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778fd30-e2c7-4020-9a09-f02739fc7aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.546645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e2b30b-082c-446e-950c-c7b02895ccb3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.548727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1778fd30-e2c7-4020-9a09-f02739fc7aeb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:00.551089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1778fd30-e2c7-4020-9a09-f02739fc7aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:00.605563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7ab9f-18ed-4110-937a-68f9fc8a2243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.606386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ab9f-18ed-4110-937a-68f9fc8a2243 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.606672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ab9f-18ed-4110-937a-68f9fc8a2243 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.606949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7ab9f-18ed-4110-937a-68f9fc8a2243 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.608408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7ab9f-18ed-4110-937a-68f9fc8a2243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.610045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7ab9f-18ed-4110-937a-68f9fc8a2243 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.617904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 20/39] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:00.619497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a91ede-f140-4497-9f74-64ee4dcc50ac None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.619796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a91ede-f140-4497-9f74-64ee4dcc50ac None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.620078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a91ede-f140-4497-9f74-64ee4dcc50ac None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.621469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 21/40] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:00.622169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a91ede-f140-4497-9f74-64ee4dcc50ac None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.648845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.663563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 20/41] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:00.664984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.665354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.665803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.698804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:00.700121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:00.700954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.754767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a91ede-f140-4497-9f74-64ee4dcc50ac None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.773963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 21/42] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:00.774784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.798244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.799235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.799539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.799815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] PATH_INFO: `/v3/domains/default/users/ca1ff62a97334431beecc4658e2e8e61/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.801994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ca1ff62a97334431beecc4658e2e8e61, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.803507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.818030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 22/43] 10.222.0.43 () {58 vars in 1361 bytes} [Fri Jun 24 08:28:00 2022] PUT /identity/v3/domains/default/users/ca1ff62a97334431beecc4658e2e8e61/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:00.818825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:00.888251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1b4757-f2e0-4b68-8d01-a554454420e5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.921446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.930728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.931599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.931892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.932348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.933005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.934413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.935312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd7d957-6c41-440c-a6c9-eb1837bb0d00 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:00.959246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 22/44] 10.222.0.43 () {60 vars in 1131 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:00.961557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:00.987812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707447e-596a-482d-b397-8b9b3c07e3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:00.988640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707447e-596a-482d-b397-8b9b3c07e3ef None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:00.988929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707447e-596a-482d-b397-8b9b3c07e3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:00.989209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707447e-596a-482d-b397-8b9b3c07e3ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:00.990454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707447e-596a-482d-b397-8b9b3c07e3ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:00.992099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707447e-596a-482d-b397-8b9b3c07e3ef None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:00.993776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 23/45] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:00.999637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:01.113462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd810cff-fb39-4cc4-b23f-87a2f516ee0d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:01.152829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ade02-b9e8-457a-8fa0-1020c6fbcfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:01.155435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ade02-b9e8-457a-8fa0-1020c6fbcfec None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:01.155435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ade02-b9e8-457a-8fa0-1020c6fbcfec None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:01.155435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ade02-b9e8-457a-8fa0-1020c6fbcfec None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:01.155435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ade02-b9e8-457a-8fa0-1020c6fbcfec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:01.156748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ade02-b9e8-457a-8fa0-1020c6fbcfec None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:01.189198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 23/46] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:00 2022] POST /identity/v3/projects => generated 312 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:02.872435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f189df0-1ce5-4519-98fe-3456bc2aa610 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.872957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f189df0-1ce5-4519-98fe-3456bc2aa610 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.873041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f189df0-1ce5-4519-98fe-3456bc2aa610 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.874377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 24/47] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:02.880346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b7f37-61dd-4f38-82a4-568d1341ee67 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:02.880784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b7f37-61dd-4f38-82a4-568d1341ee67 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:02.880918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b7f37-61dd-4f38-82a4-568d1341ee67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:02.897261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d44b7f37-61dd-4f38-82a4-568d1341ee67 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:02.898700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d44b7f37-61dd-4f38-82a4-568d1341ee67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:02.932371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:02.966122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 24/48] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:02.990019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f189df0-1ce5-4519-98fe-3456bc2aa610 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.200596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f189df0-1ce5-4519-98fe-3456bc2aa610 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.209030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78a405-f021-4ff7-ad0f-44ed5dc327bb None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.209327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78a405-f021-4ff7-ad0f-44ed5dc327bb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.209616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f78a405-f021-4ff7-ad0f-44ed5dc327bb None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.211016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 25/49] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:03.215885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.216185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.216475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.233615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:03.234248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.234799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:03.235166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.235483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.235769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.246366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:03.254181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:03.255114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:03.255932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 25/50] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:03.261012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.308558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.368001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723ecebc-836f-48de-a67f-d14c491969a3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.403295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.403969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.404277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.404540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.405298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:03.405443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 26/51] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:03.407329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.407329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.418043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 26/52] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:03.423434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.429040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.518658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.522420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459ce396-e600-475b-be40-d20f8f7aceca None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.528535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef558f5-ae21-4db4-912c-fd327cc8fdba None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.530823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.558286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.559167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.559494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.559776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.560980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.562431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.562653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.563480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.563772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.564304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.564407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 27/53] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:03.565249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.566738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.568622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 27/54] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:03.569398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.573652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.670556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1745f7-aa49-4f2b-b7e4-b24740ee3803 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.671412 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aee498e-0885-4574-9fe7-26ba3b0ddf6d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.704165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.704318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5217d734-dcea-4fdc-8bba-ade278dcc5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.704869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d734-dcea-4fdc-8bba-ade278dcc5ef None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.704985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.705156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d734-dcea-4fdc-8bba-ade278dcc5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.705253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.705435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5217d734-dcea-4fdc-8bba-ade278dcc5ef None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.705529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.706175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d734-dcea-4fdc-8bba-ade278dcc5ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.707250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.708283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5217d734-dcea-4fdc-8bba-ade278dcc5ef None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.708684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.710336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 28/55] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:03.716073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:03.736393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 28/56] 10.222.0.43 () {60 vars in 1140 bytes} [Fri Jun 24 08:28:03 2022] POST /identity/v3/projects => generated 298 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:28:03.750978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:03.820656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebf292-35d7-4f9c-8611-61fb43ca6860 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:03.856654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:03.857368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:03.857665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:03.858001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:03.859108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.domain.id=default, user.id=ca1ff62a97334431beecc4658e2e8e61) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:03.859535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:03.861349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acef983-b7b4-4c2d-b66b-2bbc0f83baeb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:03.872291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 29/57] 10.222.0.43 () {56 vars in 1322 bytes} [Fri Jun 24 08:28:03 2022] GET /identity/v3/role_assignments?user.id=ca1ff62a97334431beecc4658e2e8e61&scope.domain.id=default&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 530 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:05.752749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ca97e-f094-4b8c-a568-4faf7ff2c146 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:05.753832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ca97e-f094-4b8c-a568-4faf7ff2c146 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:05.753832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41ca97e-f094-4b8c-a568-4faf7ff2c146 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:05.755252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 29/58] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:05.760469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:05.760858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:05.761027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:05.777485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:05.778680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:05.843831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 30/59] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:05.868375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ca97e-f094-4b8c-a568-4faf7ff2c146 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:05.893405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:05.968189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41ca97e-f094-4b8c-a568-4faf7ff2c146 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:06.007513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f90d18b-19f0-4863-b4a4-a3b43ecb01ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:06.008244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d18b-19f0-4863-b4a4-a3b43ecb01ac None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:06.008529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d18b-19f0-4863-b4a4-a3b43ecb01ac None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:06.008807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f90d18b-19f0-4863-b4a4-a3b43ecb01ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:06.010003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90d18b-19f0-4863-b4a4-a3b43ecb01ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:06.011483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f90d18b-19f0-4863-b4a4-a3b43ecb01ac None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:06.013083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 30/60] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:06.018529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:06.047573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:06.118118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaa098f-98f9-4709-a3b9-8fd6f72ce8c0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:06.154832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8430a156-14fe-4ff0-a7ff-dec6f317c21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:06.155525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a156-14fe-4ff0-a7ff-dec6f317c21d None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:06.155813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a156-14fe-4ff0-a7ff-dec6f317c21d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:06.156093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8430a156-14fe-4ff0-a7ff-dec6f317c21d None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:06.156809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430a156-14fe-4ff0-a7ff-dec6f317c21d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:06.158264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8430a156-14fe-4ff0-a7ff-dec6f317c21d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:06.186048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 31/61] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:06 2022] POST /identity/v3/users => generated 275 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:09.190500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0992c7-e4a2-4be9-abbe-f4596eeef492 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.191459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0992c7-e4a2-4be9-abbe-f4596eeef492 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.191668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0992c7-e4a2-4be9-abbe-f4596eeef492 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.193162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 31/62] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:09.198213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b03890-bd05-45b7-b60b-4d0e6454c702 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.198598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b03890-bd05-45b7-b60b-4d0e6454c702 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.198789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b03890-bd05-45b7-b60b-4d0e6454c702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.225177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d8b03890-bd05-45b7-b60b-4d0e6454c702 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:09.226379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d8b03890-bd05-45b7-b60b-4d0e6454c702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:09.321221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 32/63] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:09.368756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0992c7-e4a2-4be9-abbe-f4596eeef492 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.415504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baa205-db3c-4591-ad98-0fad8c306de6 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.415836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baa205-db3c-4591-ad98-0fad8c306de6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.416096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8baa205-db3c-4591-ad98-0fad8c306de6 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.417494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 33/64] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:09.422510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5579f0-c28e-4d34-8ea5-141b1e984ae0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.422809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5579f0-c28e-4d34-8ea5-141b1e984ae0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.423089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5579f0-c28e-4d34-8ea5-141b1e984ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.460816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3b5579f0-c28e-4d34-8ea5-141b1e984ae0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:09.461951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3b5579f0-c28e-4d34-8ea5-141b1e984ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:09.489552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.528868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.540325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0992c7-e4a2-4be9-abbe-f4596eeef492 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.572119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 34/65] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:09.577406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b64051-59c3-407b-9f2c-3f16ee95ff74 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.577707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b64051-59c3-407b-9f2c-3f16ee95ff74 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.577989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b64051-59c3-407b-9f2c-3f16ee95ff74 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.579426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 35/66] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:09.584782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.585519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.585799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.586082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.588547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.588837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.589114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.607359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:09.611463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:09.612762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:09.615339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:09.616176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:09.617163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 32/67] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:09.617794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.703592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 36/68] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 08:28:09.704541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.741630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a7e79-522e-4efb-903c-feef1a566719 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.792876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.794010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.794904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.795192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.806096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.817774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:09.832846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fe4eb-a630-49af-a196-5b5ffbc7b4eb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:09.835882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:09.836775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:09.837542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 33/69] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:09.838186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.859134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:09.870712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.871429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.871883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.872166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.872927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:09.873231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:09.874640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4f61a-5803-4b6c-9ab8-9c2f2c84d439 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:09.887862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 37/70] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:09.888826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23e76-6564-46dd-9299-a9c6e12c9b64 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.889121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23e76-6564-46dd-9299-a9c6e12c9b64 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.889405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d23e76-6564-46dd-9299-a9c6e12c9b64 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.890849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 38/71] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:09.895827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d23e76-6564-46dd-9299-a9c6e12c9b64 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:09.962384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ceefbe-0a79-4e5a-8290-aef979a374c1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:09.995910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:09.996647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:09.996937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:09.997221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:09.998811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d23e76-6564-46dd-9299-a9c6e12c9b64 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.007932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:10.015551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:10.016433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:10.017192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 34/72] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:10.018084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.039062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.039781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.040072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.040359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.041118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:10.041434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.042828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d0196-15bd-4288-96f1-a85b595772fd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.047767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.052305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 39/73] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.056891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8d443-a94c-41f6-85ee-2c5a560c4771 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.057184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8d443-a94c-41f6-85ee-2c5a560c4771 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.057465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e8d443-a94c-41f6-85ee-2c5a560c4771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.073621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-51e8d443-a94c-41f6-85ee-2c5a560c4771 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:10.074832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-51e8d443-a94c-41f6-85ee-2c5a560c4771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:10.094027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.123780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cee8a-2c3b-4d71-bf8a-8890fbfedab8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.140962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 40/74] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:10.142259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8171-4886-4b79-bd4b-25deb260433d None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.142554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8171-4886-4b79-bd4b-25deb260433d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.142841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d8171-4886-4b79-bd4b-25deb260433d None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.144330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 41/75] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:10.144886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8171-4886-4b79-bd4b-25deb260433d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.161125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.162046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.162340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.162621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.177745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.179293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.181057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 35/76] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.186045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.261096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d8171-4886-4b79-bd4b-25deb260433d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.284675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f81a3a-5add-4d65-a4b3-98c0ac315c00 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.291449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.297524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.298477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.298765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.299551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:10.299862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.301737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270194ea-4388-4381-b0f6-c85c269ac736 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.309619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 42/77] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.310778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.311071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.311382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.318119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.324425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.325293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.325576 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.325849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.327039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.327696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:10.328516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.328881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:10.330122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 36/78] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.330876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.394834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 43/79] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:10.395703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.441228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9295d099-7078-46ec-9c76-0170e1875537 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.474534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.475371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.475662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.475939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.486020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:10.493549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8795efcf-7f27-446b-9a1c-d9af9cb03017 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:10.494325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:10.495370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:10.496184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 37/80] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:10.496835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.520646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.532585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.533253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.533532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.533820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.561683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.601376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.602856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.603025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b328e-a2d0-4eff-8749-a20acb462872 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.636180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.637057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.637348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.637633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.638844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.640423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.642054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 38/81] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.642825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.703408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 44/82] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:10.704245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.727445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.745063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2862d0c4-7b8c-4284-9923-cd737e549d91 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.769052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.778499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.779102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.779407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.779703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.802258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dfc4e-c88d-45ba-8e20-4d004b2c2ed0 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.836944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.837739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.838027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.838304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.839504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.840970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.842585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 45/83] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.843366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.851326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.852819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.931162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.941096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f394ad0a-acf0-4475-8127-f250056049c3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:10.949843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 39/84] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:10.952089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:10.975767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:10.976440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:10.976729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:10.977012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:10.977764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:10.978074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:10.978897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:10.979501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:10.988010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 46/85] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:10.988809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.051879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56150cdf-c3c1-46c5-b740-dba65dc28a3d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.085201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.085853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.086145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.086424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.089306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd06bff-2a9a-4b3c-95ca-4fe9de54ef9a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.123899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.124749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.125032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.125312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.126478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.127941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.129642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 47/86] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.130373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.152957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.153782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.154442 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.182727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.232317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faf2fbb-dc74-4d46-ac05-dc1ee6edf1a2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.258856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 40/87] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:11.259725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.266618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.267711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.267997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.268277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.269898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.271693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.291016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 48/88] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:11 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:11.291844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.365823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3966deee-4450-4c59-94ec-fbae72d0caf5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.379556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.397332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.407854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27c813-63bf-4399-9811-f8e6e6274f16 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.408312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.409203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.409538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.409822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.411074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.412613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.414565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 41/89] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.420256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.449573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.450635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.450926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.451567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.452845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.454326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.456054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 49/90] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.457427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.543993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628a37-0e9a-487a-8059-b0295ea1e27b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.588586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9be11-0a8d-447f-9f44-8a8381f9e6c7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.597792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.598461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.598745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.599029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.599899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=c33e8a8523ff46ea85b80ad1163fbde3) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.600212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.601669 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.607959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 42/91] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/role_assignments?user.id=c33e8a8523ff46ea85b80ad1163fbde3&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.608909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.632458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.633790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.634075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.634356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.635704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.637322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.639295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 50/92] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.640102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.661408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.681063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:11.741110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e493bc51-0dbe-48dc-b8c2-45995a599177 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.801154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111198d3-9adc-4348-86fc-95ad43c806b7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:11.827897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.828575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.828866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.829152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.830000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] RBAC: Adding query filter params (role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=c33e8a8523ff46ea85b80ad1163fbde3) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:11.831304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.831872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.838751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 43/93] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/role_assignments?user.id=c33e8a8523ff46ea85b80ad1163fbde3&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:11.839519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:11.839735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:11.840402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:11.840689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:11.840965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:11.966234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:11.968842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:11.994073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfd527-4b2a-4513-af7c-8cbe58633047 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.003757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:12.035166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:12.048882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.049975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.051001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.051538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.052592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=ca1ff62a97334431beecc4658e2e8e61) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:12.053126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.054936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.062169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 44/94] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/role_assignments?user.id=ca1ff62a97334431beecc4658e2e8e61&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:12.129310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 51/95] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:12.135671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:12.253035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ce2ee-1945-466d-ab67-9eb697d6607f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.288839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f3f08-b8fb-4ef3-b988-8f7d2cd0cce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.289820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3f08-b8fb-4ef3-b988-8f7d2cd0cce0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.290017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3f08-b8fb-4ef3-b988-8f7d2cd0cce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.290299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f3f08-b8fb-4ef3-b988-8f7d2cd0cce0 None admin] PATH_INFO: `/v3/projects/9849464e4e884f4683d9b5bb1b7d887d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.291612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3f08-b8fb-4ef3-b988-8f7d2cd0cce0 None admin] RBAC: Authorizing `identity:get_project(project_id=9849464e4e884f4683d9b5bb1b7d887d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.293115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175f3f08-b8fb-4ef3-b988-8f7d2cd0cce0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.295109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 45/96] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:12 2022] GET /identity/v3/projects/9849464e4e884f4683d9b5bb1b7d887d => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:12.301464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:12.421171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce58e3b5-3125-4c73-9611-2b5e4799760b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:12.434875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:12.459249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:12.459935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:12.460220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:12.460501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:12.461339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=9849464e4e884f4683d9b5bb1b7d887d, user.id=c33e8a8523ff46ea85b80ad1163fbde3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:12.461652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:12.463086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f1a4ee-5f25-47f8-9641-ff3067ac39b0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:12.469252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 52/97] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:12 2022] GET /identity/v3/role_assignments?user.id=c33e8a8523ff46ea85b80ad1163fbde3&scope.project.id=9849464e4e884f4683d9b5bb1b7d887d&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:13.982443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809276b9-aa8d-4b64-80bb-86d32a83056f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:13.982953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809276b9-aa8d-4b64-80bb-86d32a83056f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:13.983061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809276b9-aa8d-4b64-80bb-86d32a83056f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:13.984657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 46/98] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:14.000459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.001288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.001762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.028209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:14.030024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:14.108283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 53/99] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:14.144224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809276b9-aa8d-4b64-80bb-86d32a83056f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.217028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.283988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809276b9-aa8d-4b64-80bb-86d32a83056f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.330728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bda9db-522c-4b4a-afa1-796a47efe2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.332188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda9db-522c-4b4a-afa1-796a47efe2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.332475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda9db-522c-4b4a-afa1-796a47efe2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.332757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bda9db-522c-4b4a-afa1-796a47efe2c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.345396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bda9db-522c-4b4a-afa1-796a47efe2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-c0bda9db-522c-4b4a-afa1-796a47efe2c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:14.353374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:14.354225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:14.354985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:14.357341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 47/100] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:14.363184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.379871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5da30-9f0e-4706-b2c6-dc6999584472 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.380181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5da30-9f0e-4706-b2c6-dc6999584472 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.380651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5da30-9f0e-4706-b2c6-dc6999584472 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.383112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 48/101] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:14.388362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0981a3-94b9-4ecf-8d5c-c8323a89b6b2 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.388661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0981a3-94b9-4ecf-8d5c-c8323a89b6b2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.388946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0981a3-94b9-4ecf-8d5c-c8323a89b6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.407338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9f0981a3-94b9-4ecf-8d5c-c8323a89b6b2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:14.408852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9f0981a3-94b9-4ecf-8d5c-c8323a89b6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:14.484135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 49/102] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:14.486690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac551-b498-450d-951a-dc7b39316a1b None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.486984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac551-b498-450d-951a-dc7b39316a1b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.487296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ac551-b498-450d-951a-dc7b39316a1b None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.488935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 50/103] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:14.493813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.494106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.494393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.520242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:14.521431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:14.579240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.585419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfe13c-ab8f-43bc-b058-df4df50ce5c2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.600657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.619847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 51/104] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:14.620605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.649914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.650920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.651537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.652013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.653039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:14.653538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.659811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.677538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 54/105] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:14.678753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.727070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20914b3-1d6c-46b6-99fd-4c385f5011ec None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.791926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.792742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.793030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.793310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.794524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.796004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.797776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 52/106] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:14.799450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.811078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac15441b-5164-432b-aac9-1f80edbed5b6 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.860046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.860803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.861105 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.861416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.871599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:14.923836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:14.931668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c3104-23c5-418f-b38a-9e84382539e6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:14.942111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:14.943028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:14.943821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 55/107] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:14.944462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:14.969196 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:14.970124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:14.970413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:14.970688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] PATH_INFO: `/v3/users/d126dece1a2149b893d5df9ffc13db4a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:14.985966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] RBAC: Authorizing `identity:get_user(user_id=d126dece1a2149b893d5df9ffc13db4a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:14.988033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:14.990122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 53/108] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/users/d126dece1a2149b893d5df9ffc13db4a => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:14.991249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.059591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.097511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0069828d-89f3-4342-8258-d8b23debe56c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.103514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.154011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4aed69e-4ff0-4713-b187-4fc9b6e08dff None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.157054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.157964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.158461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.158934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.160754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:15.161469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.163891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.174256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 54/109] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.176060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.207469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c693e3e5-e531-4b6f-bed4-20b1ba3ba266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.208153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e3e5-e531-4b6f-bed4-20b1ba3ba266 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.208601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e3e5-e531-4b6f-bed4-20b1ba3ba266 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.209029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c693e3e5-e531-4b6f-bed4-20b1ba3ba266 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.209894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e3e5-e531-4b6f-bed4-20b1ba3ba266 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.211563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c693e3e5-e531-4b6f-bed4-20b1ba3ba266 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.244415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 56/110] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:28:14 2022] POST /identity/v3/projects => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:15.246945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d59a4-2913-4cff-931f-85caff4b67b9 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.247728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d59a4-2913-4cff-931f-85caff4b67b9 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.248192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d59a4-2913-4cff-931f-85caff4b67b9 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.251179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 57/111] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:15.252371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d59a4-2913-4cff-931f-85caff4b67b9 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.322794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e5ca09-23e0-4c4e-92a2-1679ea8329d3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.362626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d0ac6-5779-443a-aaf3-9a0f97ceb2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.363339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d0ac6-5779-443a-aaf3-9a0f97ceb2cb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.363632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d0ac6-5779-443a-aaf3-9a0f97ceb2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.363917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d0ac6-5779-443a-aaf3-9a0f97ceb2cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.379084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.405012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d59a4-2913-4cff-931f-85caff4b67b9 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.407587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.450649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db86537-bb69-468c-9970-22af485e0349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.452008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db86537-bb69-468c-9970-22af485e0349 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.452008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db86537-bb69-468c-9970-22af485e0349 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.452173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db86537-bb69-468c-9970-22af485e0349 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.454443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db86537-bb69-468c-9970-22af485e0349 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.455501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db86537-bb69-468c-9970-22af485e0349 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.457222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d0ac6-5779-443a-aaf3-9a0f97ceb2cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.458903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 58/112] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.459021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190d0ac6-5779-443a-aaf3-9a0f97ceb2cb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.459285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a0f5-eda7-4171-9d95-c9f9ab43ff7c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.459825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a0f5-eda7-4171-9d95-c9f9ab43ff7c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.459825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2a0f5-eda7-4171-9d95-c9f9ab43ff7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.479812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0de2a0f5-eda7-4171-9d95-c9f9ab43ff7c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:15.481242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0de2a0f5-eda7-4171-9d95-c9f9ab43ff7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:15.559360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 59/113] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:15.563771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a4c03-52a3-49d2-baf0-0d08ccb4ede0 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.564070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a4c03-52a3-49d2-baf0-0d08ccb4ede0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.564358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a4c03-52a3-49d2-baf0-0d08ccb4ede0 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.565813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 60/114] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:15.566639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a4c03-52a3-49d2-baf0-0d08ccb4ede0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.600899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 55/115] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:15.602229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.602524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.602808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.630526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:15.632384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:15.678735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.707250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a4c03-52a3-49d2-baf0-0d08ccb4ede0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.716257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.728195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 56/116] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:15.729143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.757559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.758362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.758649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.758928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.760190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.761669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.767460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 61/117] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.767460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.841502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f53595c-952c-425f-bcdc-eb3fa2b925ad None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.868137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e46a3-1fb0-4dc2-9899-6c9542260d63 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:15.875742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe827b-8f5c-4fa1-aff8-5d1be27eaf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.876559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe827b-8f5c-4fa1-aff8-5d1be27eaf7e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.876848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe827b-8f5c-4fa1-aff8-5d1be27eaf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.877124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe827b-8f5c-4fa1-aff8-5d1be27eaf7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.891781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe827b-8f5c-4fa1-aff8-5d1be27eaf7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-93fe827b-8f5c-4fa1-aff8-5d1be27eaf7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:15.900300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:15.901187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:15.901968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 57/118] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:15.903258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83ec9-860c-4aec-85c2-7a7ab9c51085 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.903258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83ec9-860c-4aec-85c2-7a7ab9c51085 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.903258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b83ec9-860c-4aec-85c2-7a7ab9c51085 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.904658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 58/119] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:15.905203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b83ec9-860c-4aec-85c2-7a7ab9c51085 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.916462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:15.917642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:15.917989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:15.918421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:15.919641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:15.921087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:15.927341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 62/120] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:15.927341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:15.941699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:15.968645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.020256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b83ec9-860c-4aec-85c2-7a7ab9c51085 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.031941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43598985-6cbc-4bc3-8d7b-f53da66f79b2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.055050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.055938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.056227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.056497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.068139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:16.069523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f06b2d-108f-4ca1-89df-06f5df48bbd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.070429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f06b2d-108f-4ca1-89df-06f5df48bbd9 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.070714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f06b2d-108f-4ca1-89df-06f5df48bbd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.070998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f06b2d-108f-4ca1-89df-06f5df48bbd9 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/c33e8a8523ff46ea85b80ad1163fbde3/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.072995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f06b2d-108f-4ca1-89df-06f5df48bbd9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f490884193064b2eb8e3e7dd40558f50, user_id=c33e8a8523ff46ea85b80ad1163fbde3, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.074440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f06b2d-108f-4ca1-89df-06f5df48bbd9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:16.076047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:16.076868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:16.077647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 59/121] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:16.078876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.095577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 63/122] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:15 2022] PUT /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/c33e8a8523ff46ea85b80ad1163fbde3/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:16.100657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.100977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.101222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.128992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:16.131501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:16.204982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d65888-851a-4b79-9137-4fbbabbb8d6e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.219875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.238374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 64/123] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:16.247852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.250728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.258075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.258760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.259045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.259359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.260184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.260543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.264111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.277358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 60/124] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.277358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.376296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b557166-047e-4107-bce7-43e307f7b778 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.400900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85641ce8-8e40-4f48-80c8-899da9be09a6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.417859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.419065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.419447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.420045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.421201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] RBAC: Adding query filter params (role.id=25def067617e4e0bb35ee3f2abcdb321, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=d126dece1a2149b893d5df9ffc13db4a) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.421583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.423544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.432378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 65/125] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/role_assignments?user.id=d126dece1a2149b893d5df9ffc13db4a&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=25def067617e4e0bb35ee3f2abcdb321 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.433767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.439944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.441194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.441484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.441763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.442549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:16.442868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.444370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.453784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 61/126] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.454352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.520694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.590254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97f2ad-50f1-4f32-8867-3d43e01bd416 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.593207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.658652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e759f06-fb6c-4cc4-a1ac-fc03e7c39484 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.663731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.664630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.664912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.665190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.667569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.669050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.670816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 62/127] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.671746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.724340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.725153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.725437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.725715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.751188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:16.759043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:16.759929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:16.761107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 66/128] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:16.762033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.870179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974bf68d-35ab-47ad-b280-9dd58d4d00f9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:16.875363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:16.909774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:16.910745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:16.911040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:16.911348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:16.912665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:16.914247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:16.915916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 63/129] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:16.916673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:16.950658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb35ce0-c6ed-4ea2-b6d9-f320c02549ac None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.021626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d930d5-f39d-4f4d-abb2-4e76aa2aac07 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.033220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.058043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.058414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.058717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.059005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.059312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.060070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:17.060377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.061186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.061662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.061759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.062214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.067977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.069696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.070499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 64/130] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.071347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.072584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 67/131] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:16 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.084192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.135942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.193529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bc9b23-8bb0-4a6a-9382-36185263defe None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.234140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.235019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.235333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.235705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.237613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.239096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.241043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 65/132] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.241607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.335974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae24d5b-7aeb-4e24-8c9f-8fb70bd0c22f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.420267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba55d77-137f-4a25-88fa-d514ec44ecab None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.420559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.421426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.421716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.421810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.422000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/ca1ff62a97334431beecc4658e2e8e61/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.432231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] RBAC: Authorizing `identity:create_grant(project_id=f490884193064b2eb8e3e7dd40558f50, user_id=ca1ff62a97334431beecc4658e2e8e61, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.433741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.460902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.461791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.462097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.462435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.463876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.465423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.467068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 66/133] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.467841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.471247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 68/134] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:17 2022] PUT /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/ca1ff62a97334431beecc4658e2e8e61/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:17.480564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.585916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.620480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f5b020-cd0c-4134-99bd-51c3a215d871 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.677913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18577d1-7abe-47eb-8470-bfacb8ce04de None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.700293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36316b19-34cd-4924-a6c8-01cf6c3d84b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.701197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36316b19-34cd-4924-a6c8-01cf6c3d84b4 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.701480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36316b19-34cd-4924-a6c8-01cf6c3d84b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.701758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36316b19-34cd-4924-a6c8-01cf6c3d84b4 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/c33e8a8523ff46ea85b80ad1163fbde3/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.709041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36316b19-34cd-4924-a6c8-01cf6c3d84b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=f490884193064b2eb8e3e7dd40558f50, user_id=c33e8a8523ff46ea85b80ad1163fbde3, role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.710516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36316b19-34cd-4924-a6c8-01cf6c3d84b4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.720075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78162754-3c26-4376-b96e-4a517146d660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.721359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78162754-3c26-4376-b96e-4a517146d660 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.722010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78162754-3c26-4376-b96e-4a517146d660 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.722626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78162754-3c26-4376-b96e-4a517146d660 None admin] PATH_INFO: `/v3/projects/9849464e4e884f4683d9b5bb1b7d887d` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.724449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78162754-3c26-4376-b96e-4a517146d660 None admin] RBAC: Authorizing `identity:get_project(project_id=9849464e4e884f4683d9b5bb1b7d887d)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.726295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78162754-3c26-4376-b96e-4a517146d660 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:17.728499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 69/135] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:17 2022] GET /identity/v3/projects/9849464e4e884f4683d9b5bb1b7d887d => generated 312 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:17.734675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5940-d50a-4f76-a5eb-190331c9fc12 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.735277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5940-d50a-4f76-a5eb-190331c9fc12 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.735903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e5940-d50a-4f76-a5eb-190331c9fc12 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.738483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 70/136] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:17.740147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5940-d50a-4f76-a5eb-190331c9fc12 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:17.748649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 67/137] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:17 2022] PUT /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/c33e8a8523ff46ea85b80ad1163fbde3/roles/f4b3603604994326b650f99b4db1e7a0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:17.750071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.750363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.750655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.795656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:17.796886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:17.860857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.930483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e5940-d50a-4f76-a5eb-190331c9fc12 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:17.964870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:17.985925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 68/138] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:17.990485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:17.991766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:17.992254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:17.992728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] PATH_INFO: `/v3/projects/9849464e4e884f4683d9b5bb1b7d887d/users/c33e8a8523ff46ea85b80ad1163fbde3/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:17.997749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9849464e4e884f4683d9b5bb1b7d887d, user_id=c33e8a8523ff46ea85b80ad1163fbde3, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:17.999762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:18.021496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 71/139] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:17 2022] PUT /identity/v3/projects/9849464e4e884f4683d9b5bb1b7d887d/users/c33e8a8523ff46ea85b80ad1163fbde3/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:18.037376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:18.185271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce5c1e-1167-4eb7-bb51-e89ae5aaadce None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:18.224624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f53c62-659e-4f31-a6e2-b04b0e84e15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:18.225448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f53c62-659e-4f31-a6e2-b04b0e84e15d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.225742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f53c62-659e-4f31-a6e2-b04b0e84e15d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.226025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f53c62-659e-4f31-a6e2-b04b0e84e15d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.227751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f53c62-659e-4f31-a6e2-b04b0e84e15d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:18.229243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f53c62-659e-4f31-a6e2-b04b0e84e15d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:18.231112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 69/140] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:18.236478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:18.367281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4241bb9b-e7d1-4d5b-8260-c45d6054acd1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:18.407646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbdc4ed-fd3d-484e-a9ad-f414dffc1456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:18.408330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc4ed-fd3d-484e-a9ad-f414dffc1456 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.408619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc4ed-fd3d-484e-a9ad-f414dffc1456 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.408900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbdc4ed-fd3d-484e-a9ad-f414dffc1456 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.410255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc4ed-fd3d-484e-a9ad-f414dffc1456 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:18.411736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbdc4ed-fd3d-484e-a9ad-f414dffc1456 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:18.441987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 72/141] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:18 2022] POST /identity/v3/users => generated 283 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:18.936427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e6089-b047-4cfe-bb71-743858dcc3b5 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.936716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e6089-b047-4cfe-bb71-743858dcc3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.936986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149e6089-b047-4cfe-bb71-743858dcc3b5 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.938377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 70/142] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:18.957327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3c4d5-70a6-42d2-b3b3-b6120acd6a9f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:18.957625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3c4d5-70a6-42d2-b3b3-b6120acd6a9f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:18.957904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3c4d5-70a6-42d2-b3b3-b6120acd6a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:18.978045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-dfa3c4d5-70a6-42d2-b3b3-b6120acd6a9f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:18.979271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dfa3c4d5-70a6-42d2-b3b3-b6120acd6a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:19.026051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.067763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 73/143] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:19.109132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e6089-b047-4cfe-bb71-743858dcc3b5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.144691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.206696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6317-255a-4880-b9d8-9aa302ba86f5 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.206975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6317-255a-4880-b9d8-9aa302ba86f5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.207186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b6317-255a-4880-b9d8-9aa302ba86f5 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.208761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 74/144] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:19.218174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.218468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.218755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.248762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:19.249979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:19.276852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149e6089-b047-4cfe-bb71-743858dcc3b5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.354318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.356377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.356667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.357199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.382312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:19.394376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:19.395388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:19.396209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:19.397403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 71/145] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:19.404393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.411009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 75/146] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:19.448331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.524370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.541619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f452ed8c-ff3b-4885-8f81-fadc8f484e0a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.615150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e810b9-a782-4548-9c04-d88d251baf54 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.616845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.617541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.617831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.618114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.619421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.619421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.619608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.620661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.635036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 72/147] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:19.650152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.671903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.673056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.673550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.674016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.688487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:19.703518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:19.704419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:19.705187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:19.707587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 76/148] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:19.713850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.793570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fca9d8-ec76-4037-a7fc-0ef8ac1dff4f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.847949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.848852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.849141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.849477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.851288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.852779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.858736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 73/149] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:19.868227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5432ee4c-3657-4661-b1da-44da30e90942 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:19.869036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.910180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:19.911509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:19.912533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:19.913026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:19.913508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:19.914492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:19.914999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:19.916618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:19.927833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 77/150] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:19.933797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:19.987258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.062432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826ae79d-e791-4aab-a586-310ae29a2d53 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.077004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0ca335-0166-48b7-8b95-2eadf9bae8f7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.119240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.119240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.119240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.119240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.140045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.141468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.141956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.142518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] PATH_INFO: `/v3/users/d126dece1a2149b893d5df9ffc13db4a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.144586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] RBAC: Authorizing `identity:get_user(user_id=d126dece1a2149b893d5df9ffc13db4a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.146293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.148332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 78/151] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/users/d126dece1a2149b893d5df9ffc13db4a => generated 289 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:20.154509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.218633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.220162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.243458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.287551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.304501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb72bb-a08d-429d-be2e-a7e4becbf0af None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.353356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.354324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.354607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.354881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.359101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 74/152] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:20.360142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.361603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.363988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 79/153] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:20.375377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.375573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.532312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440b063-a182-4dae-a9ba-9ba997d02ff5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.545373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a525c9-3e7b-4044-a665-647631f590dc None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.564334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.597962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.598931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.599238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.599528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.601077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4acae5-3b1d-41fa-bd9f-5732805dfd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.601971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4acae5-3b1d-41fa-bd9f-5732805dfd33 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.602250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4acae5-3b1d-41fa-bd9f-5732805dfd33 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.602523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4acae5-3b1d-41fa-bd9f-5732805dfd33 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/d126dece1a2149b893d5df9ffc13db4a/roles/25def067617e4e0bb35ee3f2abcdb321` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.604008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.605506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.607597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 75/154] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:20.608649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4acae5-3b1d-41fa-bd9f-5732805dfd33 None admin] RBAC: Authorizing `identity:create_grant(project_id=f490884193064b2eb8e3e7dd40558f50, user_id=d126dece1a2149b893d5df9ffc13db4a, role_id=25def067617e4e0bb35ee3f2abcdb321)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.610109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4acae5-3b1d-41fa-bd9f-5732805dfd33 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.620454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.637765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 80/155] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:20 2022] PUT /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50/users/d126dece1a2149b893d5df9ffc13db4a/roles/25def067617e4e0bb35ee3f2abcdb321 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:20.639193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f20d47-2df8-487c-9113-7ccfa2cada22 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.639550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f20d47-2df8-487c-9113-7ccfa2cada22 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.639833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f20d47-2df8-487c-9113-7ccfa2cada22 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.647344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 81/156] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:20.665349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eab63-2faf-410c-97cb-274694feeadf None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.665658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eab63-2faf-410c-97cb-274694feeadf None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.665936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133eab63-2faf-410c-97cb-274694feeadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.701960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-133eab63-2faf-410c-97cb-274694feeadf None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:20.703181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-133eab63-2faf-410c-97cb-274694feeadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:20.749921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:20.795185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345afdc3-bdbf-44fa-8a8c-e88b763748ff None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:20.839997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:20.840682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.841015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.841297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.842147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=c33e8a8523ff46ea85b80ad1163fbde3) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:20.842489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:20.845760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe6e0c7-2fb3-4b9e-8560-4693e05a1391 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:20.851592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 76/157] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/role_assignments?user.id=c33e8a8523ff46ea85b80ad1163fbde3&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:20.852877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5898-09bd-4a62-b7e1-2271a897eac4 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.853169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5898-09bd-4a62-b7e1-2271a897eac4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.853508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f5898-09bd-4a62-b7e1-2271a897eac4 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.855014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 77/158] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:20.864672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:20.865001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:20.865240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:20.879281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 82/159] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:20.892294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:20.894041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:20.948730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133eab63-2faf-410c-97cb-274694feeadf None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:20.979790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.031085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 78/160] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:21.078708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.116867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133eab63-2faf-410c-97cb-274694feeadf None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.180038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dd976-5714-4e74-8760-669c76149649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.181313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dd976-5714-4e74-8760-669c76149649 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.181795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dd976-5714-4e74-8760-669c76149649 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.182279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dd976-5714-4e74-8760-669c76149649 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.205293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dd976-5714-4e74-8760-669c76149649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-179dd976-5714-4e74-8760-669c76149649 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:21.214148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:21.215135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:21.220176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:21.222390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 83/161] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:21.228577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dd976-5714-4e74-8760-669c76149649 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.243631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.331978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10695d12-f8d4-425f-9007-2e440dd0ea74 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.373164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dd976-5714-4e74-8760-669c76149649 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.395967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.396805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.397090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.397370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.416441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:21.425509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:21.426406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:21.427775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 79/162] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:21.433346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.433685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.434042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.434328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.434610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.435404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:21.435713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.437265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.446866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 84/163] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:21.452755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.495187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.600101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d16d4e-642c-4711-9686-9650354f937e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.621448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.660048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.660665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.660950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.661243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.662002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:21.662314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.663772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.672978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 80/164] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:21.678964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.697278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77497d5c-2d56-4336-8e8e-53897e126d52 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.774912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.775836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.776182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.776465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] PATH_INFO: `/v3/users/ca1ff62a97334431beecc4658e2e8e61` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.783973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] RBAC: Authorizing `identity:get_user(user_id=ca1ff62a97334431beecc4658e2e8e61)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.785997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.788042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 85/165] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/users/ca1ff62a97334431beecc4658e2e8e61 => generated 247 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:21.793411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:21.826747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ca2de-b68b-408b-a194-0e8d259bd5f3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:21.845622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:21.891921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:21.892808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:21.893093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:21.893375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:21.894729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:21.896211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:21.898195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 81/166] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:21.904314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.026054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5555b8ca-6bbf-4ee9-8e26-7fe24c2a575a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.056147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c3db8e-7d95-4efb-8d92-926485b079c1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.082062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e2816a-5ab9-4c77-bf05-b2b64940ea11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.082731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2816a-5ab9-4c77-bf05-b2b64940ea11 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.083019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2816a-5ab9-4c77-bf05-b2b64940ea11 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.083332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e2816a-5ab9-4c77-bf05-b2b64940ea11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.102381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.115980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6edc5e0-df93-442d-bef2-d20c701275e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.116656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edc5e0-df93-442d-bef2-d20c701275e3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.116945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edc5e0-df93-442d-bef2-d20c701275e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.117229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6edc5e0-df93-442d-bef2-d20c701275e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.178988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.204120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2816a-5ab9-4c77-bf05-b2b64940ea11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.205640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e2816a-5ab9-4c77-bf05-b2b64940ea11 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.224194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edc5e0-df93-442d-bef2-d20c701275e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.232811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6edc5e0-df93-442d-bef2-d20c701275e3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.398056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 82/167] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:28:22.399713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a27f8-4609-40a8-a603-d0cb1f5eaa48 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.400003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a27f8-4609-40a8-a603-d0cb1f5eaa48 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.400282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a27f8-4609-40a8-a603-d0cb1f5eaa48 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.401561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 83/168] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:22.408030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a27f8-4609-40a8-a603-d0cb1f5eaa48 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.477159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 86/169] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:22.478495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.478803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.479070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.496626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.521430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a27f8-4609-40a8-a603-d0cb1f5eaa48 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.524472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:22.525683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:22.581836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.582498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.582707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.582989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.583284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.587936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.589639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.591595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 84/170] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:22.592262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.687589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 87/171] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:22.688447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.742332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5527a5df-74a6-4b1f-bcab-9468c4b61b9e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.778034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.778901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.779193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.779501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.780699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.782169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.784128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 85/172] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:22.784686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.793184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:22.882030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd4cd0-8923-4c39-8d90-1bc1ecaeed72 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.886974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948460cd-647c-442a-9ec9-d82d648a17e0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:22.923736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.924559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.925390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.925667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.947038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:22.947658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:22.947941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:22.948219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:22.949065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] RBAC: Adding query filter params (role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=c33e8a8523ff46ea85b80ad1163fbde3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:22.949377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:22.950845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:22.953719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:22.961454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 88/173] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/role_assignments?user.id=c33e8a8523ff46ea85b80ad1163fbde3&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 608 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:22.961744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:22.962838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 86/174] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:22.963946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:22.971088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.026410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.035204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.087675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d8095-86b8-46f5-8415-7ace05d47a55 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.123763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.124432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.124716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.124994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.125843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=ca1ff62a97334431beecc4658e2e8e61) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:23.126155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.127674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3b000e-0a07-444a-ba66-77900640c700 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.134944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 87/175] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/role_assignments?user.id=ca1ff62a97334431beecc4658e2e8e61&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:23.140206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0457598-fb7c-432c-be83-9250631105b6 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.140486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0457598-fb7c-432c-be83-9250631105b6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.140768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0457598-fb7c-432c-be83-9250631105b6 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.142355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 88/176] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:23.147818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.148138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.148375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.160066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fa63d7-31a4-41de-a5a9-fce1395e8903 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.179910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:23.179910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:23.244111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.245258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.245841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.246326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.247324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:23.247832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.249606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.265726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 89/177] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:23.266299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 89/178] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:23.272704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.292106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.359628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.402648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14e1e-942b-483b-a582-42b685049d85 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.440836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.441663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.441949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.442226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.454292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:23.462418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:23.463423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:23.465453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 90/179] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:23.469767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.520882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.553174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b5f3c6-39eb-4e20-9981-cc07ee49a1f3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.580296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f20558-a27b-4db8-a283-1a629aad4ffd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.602596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.604093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2882ee94-7de6-466a-8ef3-df4c1e2d337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.604995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882ee94-7de6-466a-8ef3-df4c1e2d337a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.605279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882ee94-7de6-466a-8ef3-df4c1e2d337a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.605564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2882ee94-7de6-466a-8ef3-df4c1e2d337a None admin] PATH_INFO: `/v3/users/c33e8a8523ff46ea85b80ad1163fbde3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.607492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882ee94-7de6-466a-8ef3-df4c1e2d337a None admin] RBAC: Authorizing `identity:get_user(user_id=c33e8a8523ff46ea85b80ad1163fbde3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.608260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2882ee94-7de6-466a-8ef3-df4c1e2d337a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.610214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 90/180] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/users/c33e8a8523ff46ea85b80ad1163fbde3 => generated 275 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:23.611510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e021f-809a-4167-a870-e46e606a6f40 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.611803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e021f-809a-4167-a870-e46e606a6f40 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.612061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e021f-809a-4167-a870-e46e606a6f40 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.617212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.617901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.618345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.619091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.619235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 91/181] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:23.619754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e021f-809a-4167-a870-e46e606a6f40 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.619863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:23.620178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.621695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4f9c19-82b4-4c68-a711-9e2b1c601c09 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.630944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 91/182] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:23.636852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085081d-99da-4069-a597-94dc798676e0 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.637261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085081d-99da-4069-a597-94dc798676e0 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.637540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4085081d-99da-4069-a597-94dc798676e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.657349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4085081d-99da-4069-a597-94dc798676e0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:23.658608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4085081d-99da-4069-a597-94dc798676e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:23.732010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 92/183] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:23.732756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085081d-99da-4069-a597-94dc798676e0 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.744485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e021f-809a-4167-a870-e46e606a6f40 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.792034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c117a6-de1c-42ed-a397-1ced52a3c42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.792837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c117a6-de1c-42ed-a397-1ced52a3c42a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.792945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c117a6-de1c-42ed-a397-1ced52a3c42a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.793157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c117a6-de1c-42ed-a397-1ced52a3c42a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.830522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.854606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4085081d-99da-4069-a597-94dc798676e0 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:23.874447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c117a6-de1c-42ed-a397-1ced52a3c42a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.875488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:23.876460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c117a6-de1c-42ed-a397-1ced52a3c42a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.898862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:23.899780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.900062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.900340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] PATH_INFO: `/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.914966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] RBAC: Authorizing `identity:get_user(user_id=6e2194ebfc1b43a7b10bc14a3f6ab465)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:23.916562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:23.918479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 93/184] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465 => generated 283 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:23.919264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:23.991941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 92/185] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:23.997163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755960eb-22e5-4608-9e93-fd6c221b7a2a None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:23.997822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755960eb-22e5-4608-9e93-fd6c221b7a2a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:23.998110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755960eb-22e5-4608-9e93-fd6c221b7a2a None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:23.999794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 93/186] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:24.004955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755960eb-22e5-4608-9e93-fd6c221b7a2a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.022823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506f2bef-1c34-479e-b59e-f6d65b05f88b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.057830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.058644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.058926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.059256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.069266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.071883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:24.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:24.081774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:24.082571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 94/187] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:24.087505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.117529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755960eb-22e5-4608-9e93-fd6c221b7a2a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.124249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.157427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.158106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.158386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.158657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.203542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553721f-0302-4a5a-a625-0dffa2cea758 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.232722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.234193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.238400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008bead-61f6-4b1d-8d04-5897f3b4e347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.239376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008bead-61f6-4b1d-8d04-5897f3b4e347 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.239921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008bead-61f6-4b1d-8d04-5897f3b4e347 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.240197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008bead-61f6-4b1d-8d04-5897f3b4e347 None admin] PATH_INFO: `/v3/projects/9849464e4e884f4683d9b5bb1b7d887d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.241387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008bead-61f6-4b1d-8d04-5897f3b4e347 None admin] RBAC: Authorizing `identity:get_project(project_id=9849464e4e884f4683d9b5bb1b7d887d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.242946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008bead-61f6-4b1d-8d04-5897f3b4e347 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.249258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 95/188] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/projects/9849464e4e884f4683d9b5bb1b7d887d => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.249258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.249258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.249258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.263841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:24.265006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:24.306976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.334955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 96/189] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:24.335111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.351380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 94/190] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:24.351380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.366636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.436754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f4c3f-0eab-42ea-9393-05dbbde0ab1a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.447174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bdde4c-ffa0-4cab-bd27-feffb3c24bcb None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.470065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.470724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.471011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.471311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.472088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:24.472395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.473771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f93564a-a4a4-462c-a9e3-644e34e1ee5f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.480288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.480939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.481220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.481501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.481618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 97/191] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.482329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=9849464e4e884f4683d9b5bb1b7d887d, user.id=c33e8a8523ff46ea85b80ad1163fbde3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:24.482640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.482808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d12b9-3e52-4e2e-a69e-b1f63535a33e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.483071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d12b9-3e52-4e2e-a69e-b1f63535a33e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.483374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d12b9-3e52-4e2e-a69e-b1f63535a33e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.484092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.487748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 98/192] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:24.487748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d12b9-3e52-4e2e-a69e-b1f63535a33e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.492440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 95/193] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/role_assignments?user.id=c33e8a8523ff46ea85b80ad1163fbde3&scope.project.id=9849464e4e884f4683d9b5bb1b7d887d&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.493241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.526292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.591690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.604389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d12b9-3e52-4e2e-a69e-b1f63535a33e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.611385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f812dcbd-299b-4d0a-960c-cf839f53b4ae None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.643405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.644307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.644591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.645330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.646092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.647585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.654398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 99/194] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.654398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.655064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b58d1-6d3a-4219-a077-8ea1a6745f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.655909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b58d1-6d3a-4219-a077-8ea1a6745f8a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.656196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b58d1-6d3a-4219-a077-8ea1a6745f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.656466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b58d1-6d3a-4219-a077-8ea1a6745f8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.657692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b58d1-6d3a-4219-a077-8ea1a6745f8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.659197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b58d1-6d3a-4219-a077-8ea1a6745f8a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.660873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 96/195] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.662079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.662322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.662602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.679809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:24.681017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:24.756108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 97/196] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:24.756956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.764416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2371496-af3b-4a2a-b6d7-ab0aa950b37c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.791482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.798676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.799577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.799883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.800159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] PATH_INFO: `/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.801337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] RBAC: Authorizing `identity:get_user(user_id=6e2194ebfc1b43a7b10bc14a3f6ab465)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.802777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.804434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 100/197] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.805240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.832469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:24.867452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d97fbf-44b5-4744-b0a1-3df08aeb28ec None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.902421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.903072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.903390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.903600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018544f-53cb-4aac-99fe-8d3026ef7e9a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:24.903711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.904478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=6e2194ebfc1b43a7b10bc14a3f6ab465) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:24.904789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.906239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.911592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 98/198] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/role_assignments?user.id=6e2194ebfc1b43a7b10bc14a3f6ab465&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:24.912357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:24.955039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:24.955767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:24.956684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:24.956968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:24.957706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:24.959252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:24.999248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 101/199] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:24 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:24.999248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.035822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.053961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5579a0d6-fd54-4059-823a-d87ce228d01f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.106098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.106882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.108178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.108467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.108763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.127591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:25.133055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988737a-6f51-44af-9b17-ba42e8fde77d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:25.138829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:25.139795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:25.140606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 99/200] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:25.147421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.186063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.187455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.188015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.188515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.279421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.281195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.286639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a293464-bf1e-4909-ac01-6f0a660344fb None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.332352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.333340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.334191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.334191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.334722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:25.335027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.336524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.349644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 100/201] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:25.355761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.372971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.398747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.425195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 102/202] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:25.431638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.491388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe9fb8-acd7-43ae-930d-6fd469139ca0 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.560834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb48c4e0-67ba-4bc6-99a7-d45754efcdb7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.571923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.572813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.573090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.573360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] PATH_INFO: `/v3/users/d126dece1a2149b893d5df9ffc13db4a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.574616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] RBAC: Authorizing `identity:get_user(user_id=d126dece1a2149b893d5df9ffc13db4a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.576082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.577725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 101/203] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/users/d126dece1a2149b893d5df9ffc13db4a => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:25.583037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.603160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.604047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.604435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.604709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.608179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.609795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.611827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 103/204] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:25.619167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:25.691080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.733301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:25.748848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb84cdf-7080-45bc-91eb-eb19b8a8de53 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.784411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d134391f-57cb-422e-9484-88ef1cfee0f2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:25.797241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.797912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.798198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.798478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.799350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] RBAC: Adding query filter params (role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=6e2194ebfc1b43a7b10bc14a3f6ab465) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:25.799705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.801170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:25.817147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 104/205] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/role_assignments?user.id=6e2194ebfc1b43a7b10bc14a3f6ab465&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:25.830770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:25.832045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:25.832330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:25.832608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:25.910564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:25.912155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.001385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.030988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 102/206] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:28:26.036986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.179043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fa961-9eff-4cb7-8df7-40b8ba747112 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.194358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:26.223242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566a9553-8bfc-4189-899c-c16f1a0b5895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.224137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a9553-8bfc-4189-899c-c16f1a0b5895 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.224422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a9553-8bfc-4189-899c-c16f1a0b5895 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.224702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566a9553-8bfc-4189-899c-c16f1a0b5895 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.232262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a9553-8bfc-4189-899c-c16f1a0b5895 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.233785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566a9553-8bfc-4189-899c-c16f1a0b5895 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.235900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 105/207] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:26.242495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:26.383908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ab1123-3fe4-4c75-9881-01d873f55532 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:26.441362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:26.442034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:26.442324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:26.442604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:26.443477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] RBAC: Adding query filter params (role.id=25def067617e4e0bb35ee3f2abcdb321, scope.project.id=f490884193064b2eb8e3e7dd40558f50, user.id=d126dece1a2149b893d5df9ffc13db4a) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:26.443794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:26.445255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d51e4f-266c-40c0-8238-c59bd7003680 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:26.456557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 103/208] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:26 2022] GET /identity/v3/role_assignments?user.id=d126dece1a2149b893d5df9ffc13db4a&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=25def067617e4e0bb35ee3f2abcdb321 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:27.332223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c564f-7d72-4415-aaf4-cb83f9022a7f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.332729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c564f-7d72-4415-aaf4-cb83f9022a7f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.332835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542c564f-7d72-4415-aaf4-cb83f9022a7f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.334284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 106/209] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:27.339570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9ac78-5037-4b43-8f9d-76f3c71ca4fb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.339967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9ac78-5037-4b43-8f9d-76f3c71ca4fb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.340152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca9ac78-5037-4b43-8f9d-76f3c71ca4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.357793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0ca9ac78-5037-4b43-8f9d-76f3c71ca4fb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:27.358993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0ca9ac78-5037-4b43-8f9d-76f3c71ca4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:27.396098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:27.429412 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 104/210] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:27.451893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c564f-7d72-4415-aaf4-cb83f9022a7f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:27.488190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518b1ef-f2cf-4c67-bb4d-0414559210b1 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.489281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518b1ef-f2cf-4c67-bb4d-0414559210b1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.489632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8518b1ef-f2cf-4c67-bb4d-0414559210b1 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.491266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 105/211] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:27.497059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.497576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.498079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.520318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:27.521902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:27.579784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542c564f-7d72-4415-aaf4-cb83f9022a7f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:27.599122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 106/212] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:27.621465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bb0ef-5a9a-4df1-a338-501c65f99797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:27.622742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb0ef-5a9a-4df1-a338-501c65f99797 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.623281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb0ef-5a9a-4df1-a338-501c65f99797 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.623387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:27.623896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb0ef-5a9a-4df1-a338-501c65f99797 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.643112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bb0ef-5a9a-4df1-a338-501c65f99797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b96bb0ef-5a9a-4df1-a338-501c65f99797 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:27.652280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:27.653213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:27.653996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:27.654850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 107/213] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:27.657096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d4ca-7047-4828-9f15-aefe5657eaa1 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.657388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d4ca-7047-4828-9f15-aefe5657eaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.657670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5001d4ca-7047-4828-9f15-aefe5657eaa1 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.660663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 108/214] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:27.663047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001d4ca-7047-4828-9f15-aefe5657eaa1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:27.717354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:27.738605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353475aa-36bc-4b22-b65f-29ef235b7dde None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:27.740398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:27.776884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9ce0e6-526c-4d5c-a4cf-5b7963b4e3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:27.777771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce0e6-526c-4d5c-a4cf-5b7963b4e3df None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.778063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce0e6-526c-4d5c-a4cf-5b7963b4e3df None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.778306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5001d4ca-7047-4828-9f15-aefe5657eaa1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:27.778423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9ce0e6-526c-4d5c-a4cf-5b7963b4e3df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.779558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ce0e6-526c-4d5c-a4cf-5b7963b4e3df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:27.781016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9ce0e6-526c-4d5c-a4cf-5b7963b4e3df None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:27.782657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 107/215] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:27.783850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.784175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.784453 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.802315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:27.803527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:27.816277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:27.816946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.817225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.817506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.818240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:27.818724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:27.819948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:27.828790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 109/216] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:27.829569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:27.877231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 108/217] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:27.883567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:27.941433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a803b-1983-44c8-b730-42f0563acc6d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:27.943913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:27.976999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:27.977590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:27.977871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:27.978145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:27.978910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:27.980332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:27.992967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:27.996321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d19d82-8352-4b79-90a3-98d11945f8ab None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.015244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 110/218] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:27 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:28.015244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.036837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.037842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.038172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.038447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] PATH_INFO: `/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.039763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] RBAC: Authorizing `identity:get_user(user_id=6e2194ebfc1b43a7b10bc14a3f6ab465)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.041249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.043507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 109/219] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:27 2022] GET /identity/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465 => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:28.048760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.161798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66387790-2164-43f1-ad0b-58d0d66ffc91 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.172931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4e69a1-332e-4299-a5ba-ce1547cc49a3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.207372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.208209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.208481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.209036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.217155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.217625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.217944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.218226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.219775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.220952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.221742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:28.222944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 110/220] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:28.228601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:28.230583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:28.231475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:28.232311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 111/221] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:28.239882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.267518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.269250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.350795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec3369-557d-4eb1-b901-0c5da091f912 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.365254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3950bd-0d84-4e83-8f09-83337ab504c2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.393450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03190fba-44ab-4f00-a2b4-bebee28d2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.394345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03190fba-44ab-4f00-a2b4-bebee28d2324 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.394626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03190fba-44ab-4f00-a2b4-bebee28d2324 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.394898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03190fba-44ab-4f00-a2b4-bebee28d2324 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849/users/6e2194ebfc1b43a7b10bc14a3f6ab465/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.396921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03190fba-44ab-4f00-a2b4-bebee28d2324 None admin] RBAC: Authorizing `identity:create_grant(project_id=58260e912d5d42f7b5b53106290e3849, user_id=6e2194ebfc1b43a7b10bc14a3f6ab465, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.398380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03190fba-44ab-4f00-a2b4-bebee28d2324 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.402690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.403314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.403597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.403880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.404728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:28.405045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.406542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.414333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 111/222] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:28 2022] PUT /identity/v3/projects/58260e912d5d42f7b5b53106290e3849/users/6e2194ebfc1b43a7b10bc14a3f6ab465/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:28.415886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 112/223] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:28.416378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972d765-4efb-4830-b27d-55ce71334cfc None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.416932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972d765-4efb-4830-b27d-55ce71334cfc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.417529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6972d765-4efb-4830-b27d-55ce71334cfc None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.419113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 112/224] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:28.424399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.425844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.426371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.426875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.446164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:28.447772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:28.519722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.524338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 113/225] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:28.547353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.568877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd87a2-7924-4e41-b22c-97607c7538c1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.576898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.657374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df58d7-0359-4c52-9708-68d5dbc56038 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.683946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.684844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.685135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.685415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] PATH_INFO: `/v3/users/18ee7e695c354a4596ea50753c9bc9e6` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.698690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.699618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.699885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.700159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.700438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] RBAC: Authorizing `identity:get_user(user_id=18ee7e695c354a4596ea50753c9bc9e6)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.702636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.704496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 113/226] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/users/18ee7e695c354a4596ea50753c9bc9e6 => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:28.710108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.712428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:28.722464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:28.723423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:28.724246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:28.724861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 114/227] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:28.731410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.834685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c45633-3119-4d5e-aea3-b125b547b3bb None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.849386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:28.872738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.873409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.873691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.873962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:28.875046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:28.875383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:28.877053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:28.886426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 115/228] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:28.892577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:28.894622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef3f9bb-4199-4ad4-8c08-8f172c142b14 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:28.987242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:28.988006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:28.988295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:28.988580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.001405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b372fbc-7848-4c80-a2c6-5f72ffcb90a7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.009435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.070894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.072195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.072790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.073361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] PATH_INFO: `/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.077980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] RBAC: Authorizing `identity:get_user(user_id=6e2194ebfc1b43a7b10bc14a3f6ab465)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.079763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.082415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 116/229] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:29.088531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.089969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.092613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.128570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.207197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 114/230] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:29.209020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.254704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.323630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33890ffe-39c8-4393-a271-d1c90abfb107 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.365746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f50d34-20c6-4871-9452-da780fd6db41 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.369835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.370703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.370978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.371274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.376930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.379090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.380933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 115/231] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:29.391784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.431199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.432248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.432537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.432814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.434291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.435769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.438086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 117/232] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:29.444228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:29.540171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.561365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e5fc37-c23a-4f1d-b85f-d5f749c7078d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.610120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb56ee-8dcb-4b0e-a15e-c41ff60cd740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.611037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb56ee-8dcb-4b0e-a15e-c41ff60cd740 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.611347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb56ee-8dcb-4b0e-a15e-c41ff60cd740 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.611631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb56ee-8dcb-4b0e-a15e-c41ff60cd740 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849/users/6e2194ebfc1b43a7b10bc14a3f6ab465/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.618453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baae8089-d530-4f1c-ab38-3f4f3d14f5e9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:29.624027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb56ee-8dcb-4b0e-a15e-c41ff60cd740 None admin] RBAC: Authorizing `identity:create_grant(project_id=58260e912d5d42f7b5b53106290e3849, user_id=6e2194ebfc1b43a7b10bc14a3f6ab465, role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.625731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dceb56ee-8dcb-4b0e-a15e-c41ff60cd740 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.628829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:29.658798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:29.659556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:29.659847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:29.660132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:29.660977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=18ee7e695c354a4596ea50753c9bc9e6) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:29.661296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:29.662975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5428d6b-bb8f-4285-8aec-60ada25ec8c7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:29.668466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 118/233] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:29 2022] PUT /identity/v3/projects/58260e912d5d42f7b5b53106290e3849/users/6e2194ebfc1b43a7b10bc14a3f6ab465/roles/f4b3603604994326b650f99b4db1e7a0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:28:29.668734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 116/234] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:29 2022] GET /identity/v3/role_assignments?user.id=18ee7e695c354a4596ea50753c9bc9e6&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:30.458295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef434-b64c-4c86-ae79-d910f34a8243 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.458812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef434-b64c-4c86-ae79-d910f34a8243 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.458812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cef434-b64c-4c86-ae79-d910f34a8243 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.460481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 119/235] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:30.465741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa8dc-c4db-4318-9301-c06e3673a3ed None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.466137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa8dc-c4db-4318-9301-c06e3673a3ed None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.466334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bfa8dc-c4db-4318-9301-c06e3673a3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.487248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-d0bfa8dc-c4db-4318-9301-c06e3673a3ed None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:30.488458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d0bfa8dc-c4db-4318-9301-c06e3673a3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:30.589527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 117/236] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:30.631430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef434-b64c-4c86-ae79-d910f34a8243 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.726226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b7761-625f-47db-95ad-04c95da84e09 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.726514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b7761-625f-47db-95ad-04c95da84e09 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.726697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b7761-625f-47db-95ad-04c95da84e09 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.728111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 118/237] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:30.734349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.734632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.734908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.764193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cef434-b64c-4c86-ae79-d910f34a8243 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:30.772173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:30.773355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:30.817016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a1594-b381-4d14-8feb-11046487273d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:30.817877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a1594-b381-4d14-8feb-11046487273d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:30.818166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a1594-b381-4d14-8feb-11046487273d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:30.818443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778a1594-b381-4d14-8feb-11046487273d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:30.821239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a1594-b381-4d14-8feb-11046487273d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:30.822699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778a1594-b381-4d14-8feb-11046487273d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:30.827717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 120/238] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:30.833683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a1594-b381-4d14-8feb-11046487273d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.871596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:30.889789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 119/239] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:30.913796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:30.951964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:30.996013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778a1594-b381-4d14-8feb-11046487273d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.025603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee327833-5277-445b-a0ac-42c881b8885f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.035082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e530f1c-3e31-4682-ad05-88d7bfcc72e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.035960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e530f1c-3e31-4682-ad05-88d7bfcc72e6 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.036344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e530f1c-3e31-4682-ad05-88d7bfcc72e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.036725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e530f1c-3e31-4682-ad05-88d7bfcc72e6 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.037552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e530f1c-3e31-4682-ad05-88d7bfcc72e6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.039107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e530f1c-3e31-4682-ad05-88d7bfcc72e6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.062174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 121/240] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:30 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:31.062437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.063337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.063626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.063905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.064164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19d22-8ff9-44c7-9d66-82138deef67e None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.064605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19d22-8ff9-44c7-9d66-82138deef67e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.064984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb19d22-8ff9-44c7-9d66-82138deef67e None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.071895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 122/241] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:31.073796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.074285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.074771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.077999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:31.086819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:31.087959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:31.088783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:31.092518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 120/242] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:31.101484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:31.103169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:31.104309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.214295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.224804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.229257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61f09a-8cf3-461f-a333-6de970b0bcf9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.233582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 123/243] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:31.267645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.268360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.268651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.268933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.269769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.270234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.271528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.272348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.281156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 121/244] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:31.292317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.430994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d29f8-f0bf-4116-baa2-d06df0ab6997 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.449788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc8787-3f63-4cff-ad22-44d38aeeefa8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.479346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.480185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.480902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.481195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.489726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.491144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.491674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.492146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] PATH_INFO: `/v3/users/f2ed9248d2904519803ec47a96bb392c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.493942 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:31.503650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:31.504586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:31.505402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 124/245] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:31.512090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.512728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] RBAC: Authorizing `identity:get_user(user_id=f2ed9248d2904519803ec47a96bb392c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.514519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.516787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 122/246] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/users/f2ed9248d2904519803ec47a96bb392c => generated 297 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:31.522386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.539825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.582218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.628480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62391b44-e6bf-4816-8b43-79a501609ef4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.628673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7e1cba-3f19-4a1d-a65f-e49aadc6e5cb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.666689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.667373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.667999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.668403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.668701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.668803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.669017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.669464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.669825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:31.670140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.671679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.682683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 125/247] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:31.688229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.743122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.744958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.765026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.801252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4fc22-364d-4d9b-92f1-9d3ced344155 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.825312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:31.842793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:31.843957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:31.844390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:31.844815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] PATH_INFO: `/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:31.846211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] RBAC: Authorizing `identity:get_user(user_id=6e2194ebfc1b43a7b10bc14a3f6ab465)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:31.847881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:31.849980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 126/248] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465 => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:31.855545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.858102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 123/249] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:31.859130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:31.968096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24bbcd4-ddf5-40f8-be7a-b1c92e443a29 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.968593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ecaa53-ec8e-4f6a-964b-c3cf8ee3fa9a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:31.983621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.005438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.006318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.006612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.006889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.007533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.008339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.008624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.008904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.010234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.011866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.013601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 124/250] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:32.019635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.069810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.087756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.089380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.134550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aaf1ac-2462-48ad-b5f4-b0a6f6a121ab None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.193846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.194521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.194804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.195082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.196198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] RBAC: Adding query filter params (role.id=25def067617e4e0bb35ee3f2abcdb321, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=f2ed9248d2904519803ec47a96bb392c) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:32.196514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.199703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060e14e6-1ff2-4d37-8f01-26c8f932b152 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.210548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 125/251] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/role_assignments?user.id=f2ed9248d2904519803ec47a96bb392c&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=25def067617e4e0bb35ee3f2abcdb321 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:32.211920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67091d6-08b6-4793-94f4-3ec3b23ea91d None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.212272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67091d6-08b6-4793-94f4-3ec3b23ea91d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.212554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67091d6-08b6-4793-94f4-3ec3b23ea91d None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.213821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 126/252] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:32.224813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.225348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.225779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.260438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:32.261943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:32.336599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 127/253] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:32.342127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.355653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.405791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 127/254] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:32.441755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.518758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.542232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb30612-71c7-41aa-a84b-ec417001c47e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.592131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcff8db9-940a-4bb2-87ec-552c57edceff None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.594571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6664f235-894c-40f8-b23d-df621d501127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.595473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664f235-894c-40f8-b23d-df621d501127 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.595758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664f235-894c-40f8-b23d-df621d501127 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.596034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6664f235-894c-40f8-b23d-df621d501127 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.597967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664f235-894c-40f8-b23d-df621d501127 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.599441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6664f235-894c-40f8-b23d-df621d501127 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.602072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 128/255] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:32.608232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1d08-9a62-45f5-bd44-24274e66da62 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.608517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1d08-9a62-45f5-bd44-24274e66da62 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.608796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af1d08-9a62-45f5-bd44-24274e66da62 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.610142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 129/256] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:32.610862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1d08-9a62-45f5-bd44-24274e66da62 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.660747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94b25b-f645-4dca-898e-982ac986ae86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.661555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94b25b-f645-4dca-898e-982ac986ae86 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.661839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94b25b-f645-4dca-898e-982ac986ae86 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.662176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94b25b-f645-4dca-898e-982ac986ae86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.674535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94b25b-f645-4dca-898e-982ac986ae86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-df94b25b-f645-4dca-898e-982ac986ae86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:32.684246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:32.685156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:32.685976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:32.687095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 128/257] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:32.688220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.688220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.688220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.715097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:32.717130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:32.779953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.809459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af1d08-9a62-45f5-bd44-24274e66da62 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:32.845667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:32.847231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:32.847714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:32.848254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:32.849321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=6e2194ebfc1b43a7b10bc14a3f6ab465) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:32.849853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:32.851832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:32.861193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 130/258] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/role_assignments?user.id=6e2194ebfc1b43a7b10bc14a3f6ab465&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:32.862939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.877362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 129/259] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:32.895643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:32.926172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:32.976191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4464d2-911c-428c-8a3c-854aa17706d5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.012952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.013791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.014174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.014635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.015625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:33.016034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.017506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.028087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 131/260] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:33.036385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.107876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c3a510-7a4d-4945-8f90-b1e4299996ac None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.191619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.196034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.196962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.197166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.197444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.198742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f669ee3-e9eb-40c7-a0cb-f8cc76fe0a23 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.217884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:33.228404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:33.229304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:33.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 130/261] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:33.236148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.238448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.239532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.239917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.240203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] PATH_INFO: `/v3/users/18ee7e695c354a4596ea50753c9bc9e6` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.241465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] RBAC: Authorizing `identity:get_user(user_id=18ee7e695c354a4596ea50753c9bc9e6)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.243782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.245734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 132/262] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/users/18ee7e695c354a4596ea50753c9bc9e6 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:33.251388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.263689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.346900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6316dd-480c-4b97-a542-515fa2138414 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.366524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ba7c6-9424-458d-b885-4b933dcfeff9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.397029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.397722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.398011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.398291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.399347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:33.399661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.401066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.404480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.405352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.405636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.405911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.407146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.408643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.415317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 133/263] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:33.416627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.419591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 131/264] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:33.420798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.443904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.519874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.533379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616453ad-a5ab-485f-82e7-ddcf4f42eb33 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.535627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e8410-e8f0-4052-9058-89ee007b4e8b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.574688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.575597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.575882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.576156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] PATH_INFO: `/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.576516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.577415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] RBAC: Authorizing `identity:get_user(user_id=6e2194ebfc1b43a7b10bc14a3f6ab465)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.577894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.578365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.578831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849/users/18ee7e695c354a4596ea50753c9bc9e6/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.578944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.580871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 132/265] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/users/6e2194ebfc1b43a7b10bc14a3f6ab465 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:33.581552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] RBAC: Authorizing `identity:create_grant(project_id=58260e912d5d42f7b5b53106290e3849, user_id=18ee7e695c354a4596ea50753c9bc9e6, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.583270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.586530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:33.599837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 134/266] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:33 2022] PUT /identity/v3/projects/58260e912d5d42f7b5b53106290e3849/users/18ee7e695c354a4596ea50753c9bc9e6/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:28:33.696528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c8742-71f5-4f09-a97a-093a726a87c2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:33.733953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:33.734615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:33.734906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:33.735253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:33.767594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:33.808830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:33.810341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:33.941292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 133/267] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:33.947168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.084771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.123149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d387bc5-57e1-4c77-b6a4-d07ff342c52b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.177751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e2a60e-d08a-4324-9c24-ff6d7e3e1ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.179000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a60e-d08a-4324-9c24-ff6d7e3e1ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.179504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a60e-d08a-4324-9c24-ff6d7e3e1ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.180098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e2a60e-d08a-4324-9c24-ff6d7e3e1ad8 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.183231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2a60e-d08a-4324-9c24-ff6d7e3e1ad8 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.183231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e2a60e-d08a-4324-9c24-ff6d7e3e1ad8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.184593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 135/268] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:33 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:34.190630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.278221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.344076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1bb033-071d-4897-aaac-ea7000fafe4f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.382885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.383590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.383874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.384151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.384997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] RBAC: Adding query filter params (role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=6e2194ebfc1b43a7b10bc14a3f6ab465) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:34.385310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.386779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb697d-ebcf-4d93-955f-8c87b1bf6860 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.393940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 134/269] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/role_assignments?user.id=6e2194ebfc1b43a7b10bc14a3f6ab465&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:34.538892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa62e58-7666-4c50-8acc-44732419d8f8 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.539413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa62e58-7666-4c50-8acc-44732419d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.539413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa62e58-7666-4c50-8acc-44732419d8f8 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.540934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 136/270] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:34.546764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.547409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.547893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.551505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbffb3-f574-490a-98e5-4d4f7455979d None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.552057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbffb3-f574-490a-98e5-4d4f7455979d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.552574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dbffb3-f574-490a-98e5-4d4f7455979d None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.554557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 137/271] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:34.560357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.560949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.561427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.585201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:34.588204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:34.594461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:34.596050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:34.682583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 135/272] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:34.692331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 138/273] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:34.709440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.717303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.763625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.801216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:34.831912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e140aa42-d8ca-48ea-be6e-687dd3ad5117 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.832109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac71c0-fc22-4b7a-8ff0-004dfed9625e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:34.875387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.876875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.877163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.877445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.884041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:34.884904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:34.885193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:34.885471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:34.888610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:34.890088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:34.891228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:34.892515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 136/274] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:34.898297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:34.899578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:34.900468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:34.901322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 139/275] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:34.907692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.030523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a6376-610a-4a61-82cc-53bfd60afaba None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.040384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11cfc1a-da0d-44a8-9dcd-c43fe0f86507 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.047796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.067524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54af34-cffe-48d2-a428-7793112b837d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.068220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54af34-cffe-48d2-a428-7793112b837d None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.068593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54af34-cffe-48d2-a428-7793112b837d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.068941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54af34-cffe-48d2-a428-7793112b837d None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.069729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54af34-cffe-48d2-a428-7793112b837d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.071288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54af34-cffe-48d2-a428-7793112b837d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.079992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.080883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.081179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.081463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.082468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:35.082780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.084355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.087098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.094282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 137/276] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:28:34 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:35.095723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29d14-a917-4f3d-a9e7-afd2f1865e2f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.096326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29d14-a917-4f3d-a9e7-afd2f1865e2f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.096433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b29d14-a917-4f3d-a9e7-afd2f1865e2f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.097744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 138/277] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:35.105543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 140/278] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.105729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.105729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.105729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.106701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.122646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:35.123906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:35.204461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 139/279] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:35.226738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.245620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392005d-790e-4e24-99aa-c8a3ec1dbe1d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.286938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.287880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.288244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.288521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] PATH_INFO: `/v3/users/d5dae2e3e20b4eeb95d0eb272c99b9d4` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.307660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] RBAC: Authorizing `identity:get_user(user_id=d5dae2e3e20b4eeb95d0eb272c99b9d4)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.309143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.311373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 141/280] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/users/d5dae2e3e20b4eeb95d0eb272c99b9d4 => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.317791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.339778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d52937-1117-4172-b4ca-a55b0b5526a6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.341838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.386807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.387882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.388266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.388644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.400223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.404641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:35.415272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:35.416213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:35.417042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 140/281] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:35.422897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.444272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e21a12-5ad9-48b1-862d-057642e77b59 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.489186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.490015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.490304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.490875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.492063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, user.id=d5dae2e3e20b4eeb95d0eb272c99b9d4, scope.system=all) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:35.492063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.494000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.500073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 142/282] 10.222.0.43 () {56 vars in 1308 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/role_assignments?user.id=d5dae2e3e20b4eeb95d0eb272c99b9d4&scope.system=all&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.556402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e31afb-71e6-4901-8d81-1926b6e878ed None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.595619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.596227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.597074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.597360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.598135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:35.598445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.599921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.610602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 141/283] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:35.616349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.735539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98aafee-0b8b-461a-af22-306d9ae3ada6 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:35.765242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.792064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:35.792952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:35.793277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:35.793504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] PATH_INFO: `/v3/users/f2ed9248d2904519803ec47a96bb392c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:35.799994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] RBAC: Authorizing `identity:get_user(user_id=f2ed9248d2904519803ec47a96bb392c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:35.801469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:35.803148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 143/284] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/users/f2ed9248d2904519803ec47a96bb392c => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:35.808711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:35.861828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:35.965615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8212b8b7-c621-49be-bc5d-5612609c9a13 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.005659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4a0f59-0e5a-454a-b2d8-e5397e254bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:36.006548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a0f59-0e5a-454a-b2d8-e5397e254bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.006833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a0f59-0e5a-454a-b2d8-e5397e254bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.007117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4a0f59-0e5a-454a-b2d8-e5397e254bd4 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.008401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a0f59-0e5a-454a-b2d8-e5397e254bd4 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:36.009867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4a0f59-0e5a-454a-b2d8-e5397e254bd4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:36.011932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 142/285] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:35 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:36.017541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:36.117072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8431a-11da-4dd3-afbd-4ceaf9c36fe4 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.117362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8431a-11da-4dd3-afbd-4ceaf9c36fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.117546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8431a-11da-4dd3-afbd-4ceaf9c36fe4 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.119085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 143/286] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:36.124546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.124801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.125084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.168243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:36.169443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:36.204009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cf0271-1016-4c0a-8df0-927464fa599e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.242574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:36.243638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.243926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.244203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849/users/f2ed9248d2904519803ec47a96bb392c/roles/25def067617e4e0bb35ee3f2abcdb321` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.252114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] RBAC: Authorizing `identity:create_grant(project_id=58260e912d5d42f7b5b53106290e3849, user_id=f2ed9248d2904519803ec47a96bb392c, role_id=25def067617e4e0bb35ee3f2abcdb321)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:36.253672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:36.282698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 144/287] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:28:36 2022] PUT /identity/v3/projects/58260e912d5d42f7b5b53106290e3849/users/f2ed9248d2904519803ec47a96bb392c/roles/25def067617e4e0bb35ee3f2abcdb321 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:36.329827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 144/288] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:36.366339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:36.449616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:36.516821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b663afeb-f889-474e-875b-78f4b290a897 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.555874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:36.556682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.556968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.557248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.567832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:36.575526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:36.576403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:36.577343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 145/289] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:36.582907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:36.615567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:36.715485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5ad9e5-f47e-4180-92a2-c2646b4cb583 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.766640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:36.767739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:36.768791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:36.769262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:36.770209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:36.770715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:36.772329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:36.785494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 145/290] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:36.790979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:36.966032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae317d77-aaaa-41cb-be9e-9864e4338ded None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:36.975713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:37.016513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:37.017407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.017695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.017977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] PATH_INFO: `/v3/users/18ee7e695c354a4596ea50753c9bc9e6` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.019785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] RBAC: Authorizing `identity:get_user(user_id=18ee7e695c354a4596ea50753c9bc9e6)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:37.021224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:37.023160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 146/291] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:36 2022] GET /identity/v3/users/18ee7e695c354a4596ea50753c9bc9e6 => generated 297 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:37.028834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:37.141343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:37.173249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caf6db2-372a-44b0-ab8a-d2dede8f0f6f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:37.228030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:37.228607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.228891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.229166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.323031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:37.324800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:37.438829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:37.465024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 146/292] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:37.476340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:37.612408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341646d6-22d5-456f-9ed4-a9ac2dcefcf9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:37.657053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3d4b75-fdbc-4cea-af03-4daa463f19c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:37.657928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d4b75-fdbc-4cea-af03-4daa463f19c1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.658212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d4b75-fdbc-4cea-af03-4daa463f19c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.658490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3d4b75-fdbc-4cea-af03-4daa463f19c1 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.663941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d4b75-fdbc-4cea-af03-4daa463f19c1 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:37.665442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3d4b75-fdbc-4cea-af03-4daa463f19c1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:37.667568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 147/293] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:37.673330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:37.834416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015586ed-a4fe-4634-bf0c-35027f4b5657 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:37.846520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479196b-c46d-463b-8be7-7486b7b357d8 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.847045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479196b-c46d-463b-8be7-7486b7b357d8 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.847273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479196b-c46d-463b-8be7-7486b7b357d8 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.848699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 148/294] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:37.853819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.854126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.854389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.884708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:37.886360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:37.891021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:37.891888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.892345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.892768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.893859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=18ee7e695c354a4596ea50753c9bc9e6) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:37.894319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:37.895965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d22a2-23a1-48a8-a4e5-71dd756ac463 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:37.902723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 147/295] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/role_assignments?user.id=18ee7e695c354a4596ea50753c9bc9e6&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:37.904152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad89a8-f3b9-4bf6-9303-4e3b82537218 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.911240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad89a8-f3b9-4bf6-9303-4e3b82537218 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.911990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ad89a8-f3b9-4bf6-9303-4e3b82537218 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.913656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 148/296] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:37.925679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284eff-4a41-4ad9-a460-37f387afd7ec None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:37.926250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284eff-4a41-4ad9-a460-37f387afd7ec None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:37.926710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd284eff-4a41-4ad9-a460-37f387afd7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:37.951248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:37.958537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bd284eff-4a41-4ad9-a460-37f387afd7ec None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:37.964039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bd284eff-4a41-4ad9-a460-37f387afd7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:37.970310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 149/297] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:37.996180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.056421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.090817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 149/298] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:38.092471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a90df-a76a-48cf-b7eb-ffa316f74668 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.092869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a90df-a76a-48cf-b7eb-ffa316f74668 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.093906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a90df-a76a-48cf-b7eb-ffa316f74668 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.095476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 150/299] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:38.101270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.101921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.102402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.121746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:38.123014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:38.131683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f993508-7290-49ed-be58-7a1512f1a097 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.181468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.182200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.182480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.182752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.199727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:38.207814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:38.208699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:38.209512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:38.212121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 150/300] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:38.213109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.223281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 151/301] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:38.224408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.259601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.330605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78271a4-93de-4ba2-ab47-15a1a03a3682 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.332397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fac1f-8c2f-4afc-b7d5-e093068403c6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.342363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.366665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.367493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.367845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.368057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.369210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.370092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.370660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.370761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.371018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.371320 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.372065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.376904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.377045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 151/302] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.377045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.377182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.384681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 152/303] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.389732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.482190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e46226-a0b3-42c0-b3a7-fb454e089880 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.483633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.495487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc9f062-91b7-4dbb-84d0-09fc39a7e585 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.523108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.524199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.524478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.524752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.536352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.537042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.537324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.537598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] PATH_INFO: `/v3/groups` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.537744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:38.538285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.539755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:38.545711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:38.546750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 152/304] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:38.548040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.559681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 153/305] 10.222.0.43 () {60 vars in 1134 bytes} [Fri Jun 24 08:28:38 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:38.565179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.619404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.673631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f868783-a06d-4432-b803-ac32454a2c76 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.716029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.716918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.717199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.717474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] PATH_INFO: `/v3/users/bd259a3b0b9d46a09bd01ac3b89a76c4` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.724812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45980ec-e080-4e03-a2e5-d54bf10b6fa9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.733115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] RBAC: Authorizing `identity:get_user(user_id=bd259a3b0b9d46a09bd01ac3b89a76c4)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.734514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.737047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 153/306] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/users/bd259a3b0b9d46a09bd01ac3b89a76c4 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.742708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.769104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.770980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.772012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.772253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.772503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.773262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.774301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.775295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.785380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 154/307] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.790515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:38.867467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537266bb-ec24-4f1e-afe0-6c9aabe3c191 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.905948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.906715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.907147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.907513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.907995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:38.910376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] RBAC: Adding query filter params (role.id=25def067617e4e0bb35ee3f2abcdb321, user.id=bd259a3b0b9d46a09bd01ac3b89a76c4, scope.system=all) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:38.910974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.912575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a25c9b-1926-49d7-aff5-fbb6df6341f0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.918728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9722ea-be96-470c-9d32-224979472de2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:38.924006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 154/308] 10.222.0.43 () {56 vars in 1308 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/role_assignments?user.id=bd259a3b0b9d46a09bd01ac3b89a76c4&scope.system=all&role.id=25def067617e4e0bb35ee3f2abcdb321 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:38.925610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d1927-2461-4446-ab52-47079ada4831 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.926121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d1927-2461-4446-ab52-47079ada4831 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.926537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1d1927-2461-4446-ab52-47079ada4831 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.928125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 155/309] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:38.937058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.937699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.938177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.957779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:38.959433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:38.965056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:38.965977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:38.966245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:38.966524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] PATH_INFO: `/v3/users/d5dae2e3e20b4eeb95d0eb272c99b9d4` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:38.967857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] RBAC: Authorizing `identity:get_user(user_id=d5dae2e3e20b4eeb95d0eb272c99b9d4)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:38.969373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:38.972195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 155/310] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:38 2022] GET /identity/v3/users/d5dae2e3e20b4eeb95d0eb272c99b9d4 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:38.977994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:39.047836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 156/311] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:39.068334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:39.086123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:39.091836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7aaefd-1c40-4c7e-b9d6-d1dc42942b13 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:39.135381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:39.136417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:39.137053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:39.137408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] PATH_INFO: `/v3/system/users/d5dae2e3e20b4eeb95d0eb272c99b9d4/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:39.139721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d5dae2e3e20b4eeb95d0eb272c99b9d4, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:39.141859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:39.159343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 156/312] 10.222.0.43 () {58 vars in 1334 bytes} [Fri Jun 24 08:28:38 2022] PUT /identity/v3/system/users/d5dae2e3e20b4eeb95d0eb272c99b9d4/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:28:39.196036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405dcc1a-f675-4723-99e3-aa3d16dd8e08 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:39.239032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:39.240200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:39.240697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:39.241142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:39.254783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:39.263034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:39.264008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:39.264817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:39.271842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 157/313] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:39.274753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:39.311672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:39.395702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5a0457-e88a-412e-9b79-ec485ccd90a9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:39.431128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:39.431875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:39.432159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:39.432430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:39.433181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:39.433484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:39.434862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:39.446652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 157/314] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:39.449892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:39.530160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:39.556953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043b391-1eb3-4c56-9842-38c67bf83cac None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:39.591376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:39.592179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:39.592466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:39.592734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] PATH_INFO: `/v3/users/f2ed9248d2904519803ec47a96bb392c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:39.593921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] RBAC: Authorizing `identity:get_user(user_id=f2ed9248d2904519803ec47a96bb392c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:39.595671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:39.601789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 158/315] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:39 2022] GET /identity/v3/users/f2ed9248d2904519803ec47a96bb392c => generated 297 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:39.603133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:39.697754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:39.714955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d605b9-e644-4b7f-aeeb-607076c7c037 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:39.751904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:39.752978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:39.753466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:39.753943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:39.827193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:39.829093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:39.918591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:39.936311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 158/316] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:39.942609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:40.045518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aaaa8-8478-49f8-b617-51416391f1ac None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:40.082993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1947e55b-a7ca-4025-81b1-d2ec772a284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:40.083902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947e55b-a7ca-4025-81b1-d2ec772a284c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.084189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947e55b-a7ca-4025-81b1-d2ec772a284c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.084471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1947e55b-a7ca-4025-81b1-d2ec772a284c None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.085669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947e55b-a7ca-4025-81b1-d2ec772a284c None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:40.087115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1947e55b-a7ca-4025-81b1-d2ec772a284c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:40.093576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 159/317] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:39 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:40.094925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:40.205769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4eb53b-2c90-4317-9eb7-4000e41c5af7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:40.240998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:40.241662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.241948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.242227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.243064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] RBAC: Adding query filter params (role.id=25def067617e4e0bb35ee3f2abcdb321, scope.project.id=58260e912d5d42f7b5b53106290e3849, user.id=f2ed9248d2904519803ec47a96bb392c) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:40.243456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:40.244970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b4b4d-e7e3-4725-bc64-d0c1d8c9eac6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:40.251091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 159/318] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:28:40 2022] GET /identity/v3/role_assignments?user.id=f2ed9248d2904519803ec47a96bb392c&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=25def067617e4e0bb35ee3f2abcdb321 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:40.685705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2689c-e74c-4f93-9f3d-5bfd2a62410d None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.686105 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2689c-e74c-4f93-9f3d-5bfd2a62410d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.686277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2689c-e74c-4f93-9f3d-5bfd2a62410d None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.687842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 160/319] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:40.692702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.692992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.693262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.709578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:40.710745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:40.758144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:40.782689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 160/320] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:40.805827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2689c-e74c-4f93-9f3d-5bfd2a62410d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:40.819143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:40.918255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2689c-e74c-4f93-9f3d-5bfd2a62410d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:40.950770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e489d1e-c76c-4d99-bbf6-0d6572a70476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:40.951526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e489d1e-c76c-4d99-bbf6-0d6572a70476 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:40.951853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e489d1e-c76c-4d99-bbf6-0d6572a70476 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:40.952130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e489d1e-c76c-4d99-bbf6-0d6572a70476 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:40.953284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e489d1e-c76c-4d99-bbf6-0d6572a70476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:40.954770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e489d1e-c76c-4d99-bbf6-0d6572a70476 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:40.957258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 161/321] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:40.962458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.036417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3791-5bfd-4914-926d-8c02203b769f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.036732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3791-5bfd-4914-926d-8c02203b769f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.036976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4e3791-5bfd-4914-926d-8c02203b769f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.038491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 162/322] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:41.043107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc2b92-2b5c-4d88-9609-3b8c80c95a4c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.043421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc2b92-2b5c-4d88-9609-3b8c80c95a4c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.043728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc2b92-2b5c-4d88-9609-3b8c80c95a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.059952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-bbfc2b92-2b5c-4d88-9609-3b8c80c95a4c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:41.061082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bbfc2b92-2b5c-4d88-9609-3b8c80c95a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:41.063090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c1e609-c4b2-4c0d-a11a-52c2dc548fb1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.099603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49181944-c914-4793-9429-fca8917bd71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.100405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49181944-c914-4793-9429-fca8917bd71d None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.100573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49181944-c914-4793-9429-fca8917bd71d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.100813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49181944-c914-4793-9429-fca8917bd71d None admin] PATH_INFO: `/v3/groups` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.101924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49181944-c914-4793-9429-fca8917bd71d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.103414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49181944-c914-4793-9429-fca8917bd71d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.118788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 161/323] 10.222.0.43 () {60 vars in 1134 bytes} [Fri Jun 24 08:28:40 2022] POST /identity/v3/groups => generated 226 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:28:41.140270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 163/324] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:41.164771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49181944-c914-4793-9429-fca8917bd71d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.216350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.283066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49181944-c914-4793-9429-fca8917bd71d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.297149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f2ff2-045d-4abc-bb93-f2746d76fea4 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.297444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f2ff2-045d-4abc-bb93-f2746d76fea4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.297706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9f2ff2-045d-4abc-bb93-f2746d76fea4 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.299073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 164/325] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:41.304209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.304503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.304785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.320757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.321563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.321847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.321947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:41.322119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.323126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:41.325191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.333685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:41.341387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:41.342272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:41.343110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 162/326] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:41.352830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.414441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 165/327] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:41.438937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.462747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.466028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e4356-a5dc-41b5-84ec-d1408f6e583b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.513335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.514257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.514549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.514837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.517356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] RBAC: Adding query filter params (name=reader) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:41.517356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.517356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.528495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 163/328] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:41.534731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.571485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f32ff-3056-4fd4-bf68-55dbfe025560 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.610207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.611268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.612295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.612754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.621109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.626503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:41.636643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:41.637715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:41.638553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:41.641001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 166/329] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:41.641733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71f086-d91f-4283-a9f3-ee8c2dd998ba None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.646115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.678669 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.679685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.680031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.680585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] PATH_INFO: `/v3/users/bd259a3b0b9d46a09bd01ac3b89a76c4` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.681670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] RBAC: Authorizing `identity:get_user(user_id=bd259a3b0b9d46a09bd01ac3b89a76c4)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.683244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.685271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 164/330] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/users/bd259a3b0b9d46a09bd01ac3b89a76c4 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:41.695726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.728738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.766322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ae7182-6043-4bbd-b739-08c51a7cba76 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.812723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.813712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.814161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.814614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.816247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:41.817036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.818573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.822565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696878b-6e19-4843-bba0-244638c71b48 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:41.833377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 167/331] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:41.838410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:41.867911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:41.868796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:41.869082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:41.869364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] PATH_INFO: `/v3/system/users/bd259a3b0b9d46a09bd01ac3b89a76c4/roles/25def067617e4e0bb35ee3f2abcdb321` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:41.870885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=bd259a3b0b9d46a09bd01ac3b89a76c4, role_id=25def067617e4e0bb35ee3f2abcdb321)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:41.872903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:41.888014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 165/332] 10.222.0.43 () {58 vars in 1334 bytes} [Fri Jun 24 08:28:41 2022] PUT /identity/v3/system/users/bd259a3b0b9d46a09bd01ac3b89a76c4/roles/25def067617e4e0bb35ee3f2abcdb321 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:41.966653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:41.976951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbab98-1553-4831-96df-f1e619e748fa None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:42.015806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb5ae74-3519-40e9-83dd-4cdfaa6a52b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:42.017201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5ae74-3519-40e9-83dd-4cdfaa6a52b6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:42.017704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5ae74-3519-40e9-83dd-4cdfaa6a52b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:42.018244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5ae74-3519-40e9-83dd-4cdfaa6a52b6 None admin] PATH_INFO: `/v3/users/d5dae2e3e20b4eeb95d0eb272c99b9d4` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:42.020102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5ae74-3519-40e9-83dd-4cdfaa6a52b6 None admin] RBAC: Authorizing `identity:get_user(user_id=d5dae2e3e20b4eeb95d0eb272c99b9d4)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:42.021763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb5ae74-3519-40e9-83dd-4cdfaa6a52b6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:42.024317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 168/333] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:41 2022] GET /identity/v3/users/d5dae2e3e20b4eeb95d0eb272c99b9d4 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:42.029731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:42.144151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:42.151692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565184b-200d-4e10-a139-c49b05fe1e6e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:42.203737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:42.204417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:42.204752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:42.205038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:42.205930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] RBAC: Adding query filter params (role.id=f64a7b4bd6b0489997f2c0dda8eed424, user.id=d5dae2e3e20b4eeb95d0eb272c99b9d4, scope.system=all) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:42.206244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:42.211830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d9880c-c848-446f-87c3-8d46f7a018b6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:42.226818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 166/334] 10.222.0.43 () {56 vars in 1308 bytes} [Fri Jun 24 08:28:42 2022] GET /identity/v3/role_assignments?user.id=d5dae2e3e20b4eeb95d0eb272c99b9d4&scope.system=all&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:43.630003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057aa50-0ce0-4732-bbaf-132c66bca7ee None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.630515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057aa50-0ce0-4732-bbaf-132c66bca7ee None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.630591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057aa50-0ce0-4732-bbaf-132c66bca7ee None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.632168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 169/335] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:43.638852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.639760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.640245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.661366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:43.662879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:43.684325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba247f7-fb60-46a4-abfd-5e52965aa0ca None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.684625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba247f7-fb60-46a4-abfd-5e52965aa0ca None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.684879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba247f7-fb60-46a4-abfd-5e52965aa0ca None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.686242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 170/336] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:43.693189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:43.693470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:43.693749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:43.728244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:43.729443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:43.761932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 167/337] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:43.828037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:43.851012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 171/338] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 24 08:28:43.905770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:43.948945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:43.968610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8f657-8009-4497-b887-a9656aa93b71 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.017246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.018607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.018885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.019153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.039529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:44.047628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:44.048594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:44.049886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 168/339] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:44.053980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.056935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.115173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c15bac8-daf4-46b3-bfb8-1c9a45ded96b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.193298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.194277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.194561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.194835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.229729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:44.238853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:44.239772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:44.240573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:44.244763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 172/340] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:44.250540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.258416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75e43de-5fe9-4715-afe6-5c77a619175e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.319924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.320614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.320900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.321178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.321938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:44.322248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.323674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.333103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 169/341] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:44.339972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.345838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.472030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4418a6-d136-4626-8bd2-f72907316741 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.515890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.533369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6047c23c-470c-4776-92c1-5303cdd5c904 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.534643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.535473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.535848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.536135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.538921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:44.539263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.540738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f4aab-c3b3-49f6-8028-fdba1c63876b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.552706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 173/342] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:44.554025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8b71b-90ca-433e-ba00-41e2c526da6b None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.554311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8b71b-90ca-433e-ba00-41e2c526da6b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.554588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c8b71b-90ca-433e-ba00-41e2c526da6b None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.555840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 174/343] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:44.560273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8b71b-90ca-433e-ba00-41e2c526da6b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:44.591914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.592583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.592870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.593141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.667612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c8b71b-90ca-433e-ba00-41e2c526da6b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:44.706039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888d3ad-5506-49d2-87e0-8f88fcf4bb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:44.706717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888d3ad-5506-49d2-87e0-8f88fcf4bb23 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.707004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888d3ad-5506-49d2-87e0-8f88fcf4bb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.707320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888d3ad-5506-49d2-87e0-8f88fcf4bb23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.728169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.729584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.737828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.764994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:44.785390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888d3ad-5506-49d2-87e0-8f88fcf4bb23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:44.786928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888d3ad-5506-49d2-87e0-8f88fcf4bb23 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:44.895951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 175/344] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:44.897351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:44.897640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:44.897913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:44.917510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:44.918708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:44.996614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 176/345] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:45.002779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.019567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.036243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 170/346] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:45.042615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.140794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a5db9-f5be-42c6-8447-acd44d2b8999 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.155070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fd777b-57f2-4944-bae0-ba7a061addf7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.157530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.188226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.189322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.189750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.190207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.191576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.193147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.195346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 177/347] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:44 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.196265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.197074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e280dd5-00e9-49a2-94d3-8d8b50b738bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.198031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280dd5-00e9-49a2-94d3-8d8b50b738bb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.198382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280dd5-00e9-49a2-94d3-8d8b50b738bb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.198720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280dd5-00e9-49a2-94d3-8d8b50b738bb None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.200205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e280dd5-00e9-49a2-94d3-8d8b50b738bb None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.201815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e280dd5-00e9-49a2-94d3-8d8b50b738bb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.203858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 171/348] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.205056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fba0d-e675-44cd-82f8-43d5756ee61a None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.205340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fba0d-e675-44cd-82f8-43d5756ee61a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.205708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fba0d-e675-44cd-82f8-43d5756ee61a None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.211233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 172/349] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:45.211233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fba0d-e675-44cd-82f8-43d5756ee61a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.286430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.309970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0913dd-65c2-484f-8267-30b4577e488b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.325365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fba0d-e675-44cd-82f8-43d5756ee61a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.349771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.350810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.351268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.351686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.364981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced6ac4-e186-488c-b311-2299ed4dacab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.365542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:45.365848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6ac4-e186-488c-b311-2299ed4dacab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.366167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6ac4-e186-488c-b311-2299ed4dacab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.366443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6ac4-e186-488c-b311-2299ed4dacab None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:45.373953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:45.375087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 24 08:28:45.376149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:45.378454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 178/350] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:45.379910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.381919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced6ac4-e186-488c-b311-2299ed4dacab None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.383783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced6ac4-e186-488c-b311-2299ed4dacab None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.385835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 173/351] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.387153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.387543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.387888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.407014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:45.408468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:45.474255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.487048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 174/352] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:45.491470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.518935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb20c-19fc-4202-a445-bc2cd4b83a95 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.562667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.564135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.564624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.565305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.567026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.568718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.571650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 179/353] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.572937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.584530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.626132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ee3acd-28a2-4ddc-a4de-52fedf06fa7a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.670392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.670392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.670392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.670392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.670803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:45.670803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.671577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.681745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 175/354] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.682770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.706460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d2caf9-5d2f-47ae-b8df-4a71a7527ee8 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.728038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.751261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.752197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.752631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.753006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.753932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=f490884193064b2eb8e3e7dd40558f50) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:45.755625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:45.757287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:45.763703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 180/355] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:45.764619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.829234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a18c0-9f68-4922-91c9-31d8e23c0d1a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.863101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:45.879476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.880407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.880756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.881100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.903263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:45.905549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:45.906466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:45.907341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 176/356] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:45.909232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:45.925002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00a3c14-c09f-4be1-b31a-0686c3f0a2fb None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:45.993894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:45.995544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:45.996036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:45.996650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:45.998152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=f490884193064b2eb8e3e7dd40558f50) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:45.998751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.000456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab277bff-96a9-4391-bc4f-f79c0c42f32e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.009344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 181/357] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 260 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.010991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d989a-a5d3-47fb-b0c3-58d7e17dba17 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.011497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d989a-a5d3-47fb-b0c3-58d7e17dba17 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.011957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d989a-a5d3-47fb-b0c3-58d7e17dba17 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.014407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 182/358] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:46.015521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d989a-a5d3-47fb-b0c3-58d7e17dba17 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.031464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.051654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57c4a2-83d4-4df9-8952-55c77e5ca19e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.101655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebbf37-b7f4-4a8e-991a-b3e5deba5e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.101990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebbf37-b7f4-4a8e-991a-b3e5deba5e36 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.102205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebbf37-b7f4-4a8e-991a-b3e5deba5e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.102487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebbf37-b7f4-4a8e-991a-b3e5deba5e36 None admin] PATH_INFO: `/v3/users/bd259a3b0b9d46a09bd01ac3b89a76c4` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.104121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebbf37-b7f4-4a8e-991a-b3e5deba5e36 None admin] RBAC: Authorizing `identity:get_user(user_id=bd259a3b0b9d46a09bd01ac3b89a76c4)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.105816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebbf37-b7f4-4a8e-991a-b3e5deba5e36 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.108292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 177/359] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:28:45 2022] GET /identity/v3/users/bd259a3b0b9d46a09bd01ac3b89a76c4 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.109510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d810-b30c-4ee5-8591-f0a89a5fe46c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.109796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d810-b30c-4ee5-8591-f0a89a5fe46c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.110072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19d810-b30c-4ee5-8591-f0a89a5fe46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.131185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9a19d810-b30c-4ee5-8591-f0a89a5fe46c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:46.132944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9a19d810-b30c-4ee5-8591-f0a89a5fe46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:46.155089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d989a-a5d3-47fb-b0c3-58d7e17dba17 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.207983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.209067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.209575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.210050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.211016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:46.211551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.213226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.215270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.225717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 183/360] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.226166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 178/361] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:46.227242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.228565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aac11-71d0-4633-8011-1d97ec6d8a87 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.229010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aac11-71d0-4633-8011-1d97ec6d8a87 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.229291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4aac11-71d0-4633-8011-1d97ec6d8a87 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.230815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 179/362] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:46.231432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aac11-71d0-4633-8011-1d97ec6d8a87 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.341570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.365808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4aac11-71d0-4633-8011-1d97ec6d8a87 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.405000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb18983-7ba9-44b4-bd36-23dbdf7a2122 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.414923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.415692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.416445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.416729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.448088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.448776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.449060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.449345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.450455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] RBAC: Adding query filter params (role.id=25def067617e4e0bb35ee3f2abcdb321, user.id=bd259a3b0b9d46a09bd01ac3b89a76c4, scope.system=all) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:46.450778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.452337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2497040-2e4a-4e21-bdeb-bc78df723794 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.469287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 184/363] 10.222.0.43 () {56 vars in 1308 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/role_assignments?user.id=bd259a3b0b9d46a09bd01ac3b89a76c4&scope.system=all&role.id=25def067617e4e0bb35ee3f2abcdb321 => generated 510 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:46.470624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.470912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.471253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.496781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:46.498052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:46.531438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:46.532962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:46.591822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.612685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 185/364] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:46.614860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.692367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.702266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 180/365] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:46.703188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.798507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c76a70-cf2f-442a-9187-020f4732ff4b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.815530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc262522-4dec-4200-af4b-cbd5bf6c7cce None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:46.841322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.842148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.842437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.842711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.856011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:46.856894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:46.857129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:46.857412 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:46.858078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:46.868607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:46.869533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:46.870348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:46.872393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:46.873149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 186/366] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:46.874129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:46.885321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:46.886278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:46.887083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 181/367] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:46.887781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:46.938273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:46.955698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.008575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9274bb1b-66d3-41c1-a245-ee07fd64289c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.024092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf47ed-3eb4-46cf-8c28-fdaeb436d635 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.054132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.055540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.055987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.056795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.058620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.061027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.063388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 187/368] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.064312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.064429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.066514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.066514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.066514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.068248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:47.068677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.071035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.079933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 182/369] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.080917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.192441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f9198b-1313-4d72-9b97-a382d484533f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.210519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.241102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.242102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.242210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd56ea5-4096-4e01-8d21-acd68bbb58de None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.242662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.243124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.244372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:47.245772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.247362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.253182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.261585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 188/370] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.266193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.293687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.294553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.294839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.295114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.298502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.300834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.303251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 183/371] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.309200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.388725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f908bd-021a-4cb9-a452-b4e5574ae816 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.425325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.425966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffc67bb-85f0-43f4-8e55-53c0d5bcb410 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.426080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.426277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.426521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.469649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.470224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.470508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.470789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.478298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.508352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.510048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.544637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.575286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.576874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.640720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 189/372] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:47.641741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.704979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 184/373] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:47.706484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.743284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.773300 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b51147-6c1f-4581-a886-9093d8411be1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.795101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:47.812422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.813087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.813372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.813651 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.814580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=58260e912d5d42f7b5b53106290e3849) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:47.814941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.816525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.821534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de95db1-217f-4d43-9d29-032056709810 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:47.824503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 190/374] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.825324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.862721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:47.863669 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:47.863957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:47.864237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:47.865789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:47.867289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:47.869434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 185/375] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:47.878767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:47.968450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ea8cf-3719-4a32-b0b6-d811b777b3e3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.014202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c78622-a053-4bb3-b43d-188b0a8fcda3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.014805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.015822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.016111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.016392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] PATH_INFO: `/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.024085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] RBAC: Authorizing `identity:get_project(project_id=ffa0e0eb99bd42cebd2bc5b9d635fa5d)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:48.025683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:48.027636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 191/376] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d => generated 344 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:48.033181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.047828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.057203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.058075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.058407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.058693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.060728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:48.062291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:48.064446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 186/377] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:47 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:48.093376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.113683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.172025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ae295-fda0-4f4a-8870-781022ffbfb1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.224361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.225251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.225535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.225815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] PATH_INFO: `/v3/groups/f6edf31b4eb044dab23500187bf9a136` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.236144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] RBAC: Authorizing `identity:get_group(group_id=f6edf31b4eb044dab23500187bf9a136)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:48.237754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:48.239383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ce09bb-59fb-4581-be82-55c8cd9255e7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.239851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 192/378] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/groups/f6edf31b4eb044dab23500187bf9a136 => generated 229 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:48.245509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.282513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.283441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.283929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.284370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.285548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=58260e912d5d42f7b5b53106290e3849) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:48.286007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:48.287735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614c727b-9c82-4432-96da-885ec49872e1 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:48.294235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 187/379] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:48.295875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70137a3-9949-47ba-9164-210ecbf404e4 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.296435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70137a3-9949-47ba-9164-210ecbf404e4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.296858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70137a3-9949-47ba-9164-210ecbf404e4 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.302709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 188/380] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:48.304396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e556-f9df-476f-93b8-f2487fb8d8c8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.304838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e556-f9df-476f-93b8-f2487fb8d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.305282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520e556-f9df-476f-93b8-f2487fb8d8c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.325899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-8520e556-f9df-476f-93b8-f2487fb8d8c8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:48.327449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8520e556-f9df-476f-93b8-f2487fb8d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:48.352880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.393229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d45cb47-d505-42fb-b48b-da741f406600 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.433732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.434692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.435126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.435594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.436673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] RBAC: Adding query filter params (group.id=f6edf31b4eb044dab23500187bf9a136, role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=ffa0e0eb99bd42cebd2bc5b9d635fa5d) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:48.437154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:48.438776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:48.446450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 193/381] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/role_assignments?group.id=f6edf31b4eb044dab23500187bf9a136&scope.project.id=ffa0e0eb99bd42cebd2bc5b9d635fa5d&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 260 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:48.460639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 189/382] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:48.488986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.515721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42569845-8d87-421c-96af-2326ec29393f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.516735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42569845-8d87-421c-96af-2326ec29393f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.517160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42569845-8d87-421c-96af-2326ec29393f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.523601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 190/383] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:48.525973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.526421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.526870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.550058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.567654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:48.569097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:48.675501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20553057-a30b-4b6a-8682-dce04b434897 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.684319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 191/384] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:48.718333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.719417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.719617 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.720021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.720461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.723586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:48.735607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:48.744379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:48.745411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:48.746237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 194/385] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:48.754169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:48.915810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca171c4-a96a-4103-8905-ffdfc3e85a77 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.962339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f111fd-f726-4795-8d8a-898b8f45c095 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:48.985150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:48.985970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:48.986250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:48.986524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:48.992167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.008576 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:49.016715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:49.017636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:49.018501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 192/386] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:49.024469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.030349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.031266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.032548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.032826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.034217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:49.034526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.035929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.051247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 195/387] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.053510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.137896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.190409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a583b60-b386-4132-8642-d8d4bab2ab08 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.227509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.228376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.228655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.228928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.230117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.231602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.233552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 196/388] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.240119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.257372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec48eb65-874d-493f-aa43-9c2358d36f81 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.319911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.321477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.321761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.322040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.322797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:49.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.324533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.343379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 193/389] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.349826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.429515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb092029-c77d-4e4e-b421-4ab7cc273052 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.448295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.470333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.485316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.486639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.487130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.487669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.489055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.490725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.492962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 197/390] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.502763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.524558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75c5431-5dda-4ef0-a645-2381f387068e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.566728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.567625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.567909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.568188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.569364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.570807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.573055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 194/391] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.580223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.708302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fb071f-4ba8-4533-b43d-61818b7dfd95 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.759586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bc1ded-f2aa-496d-beaf-348226fe7b5a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:49.768627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.769520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.769803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.770080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.779837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f490884193064b2eb8e3e7dd40558f50, group_id=4a90681ec46d435e8d13bff2ac87031c, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.784209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.805139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.820463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 198/392] 10.222.0.43 () {58 vars in 1442 bytes} [Fri Jun 24 08:28:49 2022] PUT /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:49.824854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30ee770-4cdc-4b75-99d6-c8c6f06d1e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:49.825757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ee770-4cdc-4b75-99d6-c8c6f06d1e54 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:49.826040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ee770-4cdc-4b75-99d6-c8c6f06d1e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:49.826312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30ee770-4cdc-4b75-99d6-c8c6f06d1e54 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:49.827618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ee770-4cdc-4b75-99d6-c8c6f06d1e54 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:49.829111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30ee770-4cdc-4b75-99d6-c8c6f06d1e54 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:49.831942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 195/393] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:49 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:49.837629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:49.913363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:49.975463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e07816a-f01f-4c63-bc0d-df25bb6400e2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:50.018466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab80c7ed-cccd-4823-9598-d90bd021afa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:50.019413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c7ed-cccd-4823-9598-d90bd021afa8 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.019693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c7ed-cccd-4823-9598-d90bd021afa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.019973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab80c7ed-cccd-4823-9598-d90bd021afa8 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.025463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80c7ed-cccd-4823-9598-d90bd021afa8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f490884193064b2eb8e3e7dd40558f50, group_id=4a90681ec46d435e8d13bff2ac87031c, role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:50.026936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab80c7ed-cccd-4823-9598-d90bd021afa8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:50.053028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 199/394] 10.222.0.43 () {58 vars in 1442 bytes} [Fri Jun 24 08:28:49 2022] PUT /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f4b3603604994326b650f99b4db1e7a0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:50.738835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f1914-85ac-4570-bddc-5cce481113e5 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.739388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f1914-85ac-4570-bddc-5cce481113e5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.739474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f1914-85ac-4570-bddc-5cce481113e5 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.741063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 196/395] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:50.746045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbff763-06e9-47b4-8fe1-c2f19091800c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:50.746339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbff763-06e9-47b4-8fe1-c2f19091800c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:50.746617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbff763-06e9-47b4-8fe1-c2f19091800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:50.768327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0cbff763-06e9-47b4-8fe1-c2f19091800c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:50.769519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0cbff763-06e9-47b4-8fe1-c2f19091800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:50.847699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 200/396] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:50.880204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f1914-85ac-4570-bddc-5cce481113e5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.005003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f1914-85ac-4570-bddc-5cce481113e5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.019610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e1b25-0ad3-4995-b7c2-82d6da7f6fa7 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.020573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e1b25-0ad3-4995-b7c2-82d6da7f6fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.021065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e1b25-0ad3-4995-b7c2-82d6da7f6fa7 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.022674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 201/397] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:51.028461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184468-a26d-4e87-ad26-0bfc79502e99 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.029039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184468-a26d-4e87-ad26-0bfc79502e99 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.029538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01184468-a26d-4e87-ad26-0bfc79502e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.052881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-01184468-a26d-4e87-ad26-0bfc79502e99 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:51.055181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-01184468-a26d-4e87-ad26-0bfc79502e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:51.078592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.079769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.080262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.081301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.092384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:51.101047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:51.102119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:51.102977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:51.108384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 197/398] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:51.115172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.121736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.160280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 202/399] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:51.163349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5289-99c8-4a61-83c4-a60c89828ddb None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.163645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5289-99c8-4a61-83c4-a60c89828ddb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.163927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aa5289-99c8-4a61-83c4-a60c89828ddb None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.165524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 203/400] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:51.170426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.170716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.171000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.192206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:51.193632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:51.205812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.255378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ce2d6a-5b4c-4a8b-86cd-3659b08776a4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.270359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 204/401] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:51.271439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.297762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.298751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.299301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.299714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.300662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] RBAC: Adding query filter params (name=member) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:51.301120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:51.303743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:51.313967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 198/402] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:51.319477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.409999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2877ae-fb79-449b-a6e4-62e32d1ab145 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.412057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.449164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fc6f99-7693-451a-85f2-c8a305523898 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.459717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.461542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.462035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.462532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.466530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.480992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:51.489718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.490536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.490821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.491097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:51.492144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:51.493044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:51.493910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:51.500196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 205/403] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:51.500196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.524251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:51.532956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:51.533914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:51.534748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 199/404] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:51.535966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.641781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607234b1-a28d-4f88-b49c-e28245f42337 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.657750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b235c-3bd5-4864-a080-d359e290aacd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.680286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.681155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.681488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.681718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.682933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:51.684678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:51.686315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 206/405] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:51.687367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.697098 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.698112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.698632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.699132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.700140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:51.700672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:51.702309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:51.711962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 200/406] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:51.713076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.722631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.759692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.806721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4594d1f-d3df-495b-bd3f-5fcb55f07561 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.822975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893812a3-b326-4694-9135-a484d36dff51 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.843507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.844182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.844528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.844811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.845674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:51.846446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:51.847996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:51.857777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 207/407] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:51.858778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.860527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:51.862160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:51.862923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:51.863421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:51.864940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:51.866622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:51.868843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 201/408] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:51.869840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:51.962309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:51.974023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc548713-29f4-4f5e-b4dd-01d2fa66ff9a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.977310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89ce25-628e-43fc-b283-d2edb079523b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:51.984778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.013944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.014506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.014863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.015197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.015507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] PATH_INFO: `/v3/groups/f6edf31b4eb044dab23500187bf9a136` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.016403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.016759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] RBAC: Authorizing `identity:get_group(group_id=f6edf31b4eb044dab23500187bf9a136)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.017094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.017579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.018252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.020292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.020411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 208/409] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/groups/f6edf31b4eb044dab23500187bf9a136 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:52.022169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.022754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.025187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 202/410] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:51 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:52.030404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.133860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75582b51-1d0b-4d72-b12e-5b8b8c9fab03 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.134509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227ec47-d4b8-4a6a-8c32-6b50eb60432e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.171054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972180c-64a6-4344-a087-77e921804247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.171769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972180c-64a6-4344-a087-77e921804247 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.172055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972180c-64a6-4344-a087-77e921804247 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.172481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972180c-64a6-4344-a087-77e921804247 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.174530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972180c-64a6-4344-a087-77e921804247 None admin] RBAC: Authorizing `identity:create_grant(project_id=58260e912d5d42f7b5b53106290e3849, group_id=4a90681ec46d435e8d13bff2ac87031c, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.177665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5972180c-64a6-4344-a087-77e921804247 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.183269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.184421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.184831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.185203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.186579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.188156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.190313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 203/411] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:52.191347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.194802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 209/412] 10.222.0.43 () {58 vars in 1442 bytes} [Fri Jun 24 08:28:52 2022] PUT /identity/v3/projects/58260e912d5d42f7b5b53106290e3849/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:52.195952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972180c-64a6-4344-a087-77e921804247 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.224755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.252191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.301347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2e2bb-ac64-4027-aadd-cae6c32e3dd6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.347192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.348112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.348398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.348679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] PATH_INFO: `/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.352016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] RBAC: Authorizing `identity:get_project(project_id=ffa0e0eb99bd42cebd2bc5b9d635fa5d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.353523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.355593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 204/413] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:52.370566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.392348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5972180c-64a6-4344-a087-77e921804247 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.470338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43b0111-1d6e-4add-b557-d5786e3ed7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.472333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b0111-1d6e-4add-b557-d5786e3ed7a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.472618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b0111-1d6e-4add-b557-d5786e3ed7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.472902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43b0111-1d6e-4add-b557-d5786e3ed7a9 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.476099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b0111-1d6e-4add-b557-d5786e3ed7a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=58260e912d5d42f7b5b53106290e3849, group_id=4a90681ec46d435e8d13bff2ac87031c, role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.478219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43b0111-1d6e-4add-b557-d5786e3ed7a9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.499814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 210/414] 10.222.0.43 () {58 vars in 1442 bytes} [Fri Jun 24 08:28:52 2022] PUT /identity/v3/projects/58260e912d5d42f7b5b53106290e3849/groups/4a90681ec46d435e8d13bff2ac87031c/roles/f4b3603604994326b650f99b4db1e7a0 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:52.507120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03040520-74b6-4b4a-bc86-f320f70e7425 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.521877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.551929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9a54b-97ce-4d20-8e9c-58be4d78d15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.552840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a54b-97ce-4d20-8e9c-58be4d78d15d None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.553124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a54b-97ce-4d20-8e9c-58be4d78d15d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.553403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9a54b-97ce-4d20-8e9c-58be4d78d15d None admin] PATH_INFO: `/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d/groups/f6edf31b4eb044dab23500187bf9a136/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.561636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a54b-97ce-4d20-8e9c-58be4d78d15d None admin] RBAC: Authorizing `identity:create_grant(project_id=ffa0e0eb99bd42cebd2bc5b9d635fa5d, group_id=f6edf31b4eb044dab23500187bf9a136, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:52.563156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9a54b-97ce-4d20-8e9c-58be4d78d15d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:52.592227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 205/415] 10.222.0.43 () {58 vars in 1442 bytes} [Fri Jun 24 08:28:52 2022] PUT /identity/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d/groups/f6edf31b4eb044dab23500187bf9a136/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:28:52.616043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6201a3c-37a8-4e99-ade6-4f1093e220f7 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.616393 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6201a3c-37a8-4e99-ade6-4f1093e220f7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.616609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6201a3c-37a8-4e99-ade6-4f1093e220f7 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.618045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 211/416] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:52.624243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.624243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.625134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.650183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:52.651459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:52.682593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5c6d1-5d7b-4b76-84f6-89dc186fadda None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.683294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5c6d1-5d7b-4b76-84f6-89dc186fadda None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.683804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f5c6d1-5d7b-4b76-84f6-89dc186fadda None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.687035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 212/417] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:52.692797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.693097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.693525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.712359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:52.713699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:52.754778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 206/418] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:52.787444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.794774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 213/419] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:52.819629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:52.832120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.909969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:52.913744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9851c8a-3079-4b19-a5cb-671a1375242f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:52.980348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:52.981176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:52.981466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:52.981743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:52.987692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152574b8-1a9b-4974-8523-fe4a25804cd4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.012178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:53.020972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:53.021870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:53.022697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:53.027347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 207/420] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:53.028839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.035617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.036436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.036719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.036995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.060752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:53.072934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:53.074984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:53.076965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 214/421] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:28:53.083175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.184958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.224251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2d61f-1dac-41d8-ae0c-51e4c3ae61c9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.264073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc625b7-f9a5-4f0e-ae20-16783ba0cc5d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.264961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.265693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.265981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.266261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.267015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:53.269172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.270564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.284924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 215/422] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:53.287743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.352980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.354036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.354537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.355028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.356029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:53.356588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.358201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.379045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 208/423] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:53.396371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.423237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c03009-8044-4fd2-a46e-6a167cea2048 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.451964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.461475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.462496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.464783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.465221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.469164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.562259 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329147b-0350-4636-bcf3-73759600ce64 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.605666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.606321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.606601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.606880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.623197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.631253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.729759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 216/424] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:53.735290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.744754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.746258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.789914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.798656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:53.910397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 209/425] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 24 08:28:53.915995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:53.932239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da93d89-9c94-430c-a8cd-f41bf76231b5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:53.972870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:53.973746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:53.974029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:53.974305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:53.978692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:53.982574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:53.984290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 217/426] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:53.990754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.109170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.112952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ed7a1c-0624-49ca-9493-cacc3524e3b5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.122515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c13c1a-0a01-4a6b-a87e-c60772cb4b40 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.183239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.184363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.185233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.185514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.185864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.187821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.189258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.194242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 218/427] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:28:54.194865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.195734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.196017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.196288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] PATH_INFO: `/v3/projects/f490884193064b2eb8e3e7dd40558f50` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.197591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] RBAC: Authorizing `identity:get_project(project_id=f490884193064b2eb8e3e7dd40558f50)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.199031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.200742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 210/428] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:53 2022] GET /identity/v3/projects/f490884193064b2eb8e3e7dd40558f50 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:54.201697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.206052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.328563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07fb898-0991-4bf9-960a-c4d65871d89b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.365181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.366152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.366511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.366794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.368025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.369488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.371491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 211/429] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:54.381990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.447923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85e6399-2863-430a-89d1-09d1c40bee3b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.479687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.518656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.519352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.519639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.519918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.520761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=f490884193064b2eb8e3e7dd40558f50) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:54.521073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.522516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a32f3b-7260-4e08-82a5-bbc452a23654 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.538350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 219/430] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 611 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:54.551166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2861ce5-d13b-4a7a-b445-222b2913fa69 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:54.589427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:54.590002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.590285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.590565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.591434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=f490884193064b2eb8e3e7dd40558f50) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:54.591751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:54.593222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814755ed-f2e0-4f04-88ad-716b1e5bc568 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:54.600260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 212/431] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=f490884193064b2eb8e3e7dd40558f50&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:54.740426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba8ddc-c27c-4090-b1d8-c4a87bb66967 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.740803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba8ddc-c27c-4090-b1d8-c4a87bb66967 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.740980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba8ddc-c27c-4090-b1d8-c4a87bb66967 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.742786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 220/432] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:54.748048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65531aa-85ce-4d8a-8c26-fa9763bd61a6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.748407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65531aa-85ce-4d8a-8c26-fa9763bd61a6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.748615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65531aa-85ce-4d8a-8c26-fa9763bd61a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.772176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b65531aa-85ce-4d8a-8c26-fa9763bd61a6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:54.773404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b65531aa-85ce-4d8a-8c26-fa9763bd61a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:54.872207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 213/433] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:54.895231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba8ddc-c27c-4090-b1d8-c4a87bb66967 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:54.929855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ac663-ec33-4893-a1aa-82f7949cd514 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.930318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:54.930688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ac663-ec33-4893-a1aa-82f7949cd514 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.931169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ac663-ec33-4893-a1aa-82f7949cd514 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.933121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 214/434] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:54.938551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:54.939084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:54.939631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:54.957465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:54.959082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:55.014114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.015948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba8ddc-c27c-4090-b1d8-c4a87bb66967 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.043856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 215/435] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:55.062771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.063625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.063923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.064192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.069035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.076581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:55.084907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:55.085805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:28:55.086690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 221/436] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:55.092608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.202920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ba66b7-e0bf-4c4a-ac52-96320b3c2430 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.205143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.216844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab0c158-7db9-4c44-aeef-016d34277d65 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.243755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e23ec14-2f73-4e28-b79b-0795c76a7e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.244547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ec14-2f73-4e28-b79b-0795c76a7e18 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.244842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ec14-2f73-4e28-b79b-0795c76a7e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.245104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e23ec14-2f73-4e28-b79b-0795c76a7e18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.259339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e23ec14-2f73-4e28-b79b-0795c76a7e18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:55.260757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.261431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.261716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.261992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.262763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:55.263068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.265402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215855-c504-4e31-98a8-d3e21b5ea6b0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6e23ec14-2f73-4e28-b79b-0795c76a7e18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:55.267022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:55.268019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:55.268841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 216/437] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:55.270342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54c573-0bd7-4877-994a-a15e5d255190 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.270342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54c573-0bd7-4877-994a-a15e5d255190 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.270342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54c573-0bd7-4877-994a-a15e5d255190 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.271666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 217/438] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:55.274722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54c573-0bd7-4877-994a-a15e5d255190 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.276975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 222/439] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.278991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.279956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.279956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.297475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:55.298657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:55.324808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.369595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 223/440] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:55.370436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.384042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54c573-0bd7-4877-994a-a15e5d255190 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.419490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.420160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.420454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.420730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.421490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:55.421796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.423187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.431994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 218/441] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.436135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.465442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.480841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb908ae5-3d3b-4f29-8e46-a13f6abb9c1d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.514449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.515092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.515406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.515686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.534096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8213100-5d55-48bf-a5e9-80980a8289d0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.536418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.567915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.568715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.568997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.569272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.579164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:28:55.583894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.585370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:28:55.587807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:55.588767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:28:55.589572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:55.590433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 219/442] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:55.591394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.666515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.684696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 224/443] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:55.685440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.692766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb396b87-3160-49d9-ac63-39b917106860 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.726513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.727156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.727470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.727749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.757606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.780581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58e12b-12f4-4483-a185-25dfd645389d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.793746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.795238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.814615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.815320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.815607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.815885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.816626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:55.816937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.818285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.826009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 225/444] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.826786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.874459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.894572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 220/445] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:55.895451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.928847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c5c3b7-ab6c-4adb-b133-bf795d2f7312 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:55.956192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:55.962898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:55.963798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:55.964084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:55.964362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:55.965547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:55.966972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:55.968675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 226/446] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:55.969444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:55.993261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f75b-8958-45f6-b9c0-7bdef7272d7f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.026195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.026813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.027090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.027396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.066095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4b58f-9fa7-4eb6-8b15-e44e932ac8e6 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.079331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.092902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.094364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.100523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.101387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.101673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.101955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] PATH_INFO: `/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.103135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] RBAC: Authorizing `identity:get_project(project_id=ffa0e0eb99bd42cebd2bc5b9d635fa5d)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.104581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.106265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 227/447] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/projects/ffa0e0eb99bd42cebd2bc5b9d635fa5d => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.107097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.157616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.199406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 221/448] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:28:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:28:56.200253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.209902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d0299-edb0-4459-ae8b-cd6661157e84 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.243352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.244308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.244598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.244878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.246068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.247544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.249175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 228/449] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.249931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.298866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.300713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205faa0-8997-4a0d-9d1e-aadb9acfaee2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.334785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.335678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.335963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.336239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] PATH_INFO: `/v3/groups/f6edf31b4eb044dab23500187bf9a136` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.337413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] RBAC: Authorizing `identity:get_group(group_id=f6edf31b4eb044dab23500187bf9a136)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.338870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.340512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 222/450] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/groups/f6edf31b4eb044dab23500187bf9a136 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.341276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.350205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fcef4b-0e3f-4b7b-83d1-d50d0577992c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.372311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.382861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.383750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.384035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.384315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] PATH_INFO: `/v3/projects/58260e912d5d42f7b5b53106290e3849` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.385506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] RBAC: Authorizing `identity:get_project(project_id=58260e912d5d42f7b5b53106290e3849)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.386933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.388650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 229/451] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/projects/58260e912d5d42f7b5b53106290e3849 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.389405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.440128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d17bd4-b978-4129-be04-16a9a383e55b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.473086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.473935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.474217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.474490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.475343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f64a7b4bd6b0489997f2c0dda8eed424, scope.project.id=58260e912d5d42f7b5b53106290e3849) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:56.475658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.477100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.482832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 223/452] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f64a7b4bd6b0489997f2c0dda8eed424 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.483621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.489877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a2502-bccc-4cfe-9ad0-2920965b39d2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.511444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.535438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.537415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.537901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.538336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.539429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] RBAC: Adding query filter params (group.id=f6edf31b4eb044dab23500187bf9a136, role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=ffa0e0eb99bd42cebd2bc5b9d635fa5d) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:56.539911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.542326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.551641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 230/453] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/role_assignments?group.id=f6edf31b4eb044dab23500187bf9a136&scope.project.id=ffa0e0eb99bd42cebd2bc5b9d635fa5d&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.606938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:56.616006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1096b50d-e5d5-44e7-a3c7-6130cfbcdc45 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.657503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2591c00-5058-41f7-b134-f00b0a7bb965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.658426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591c00-5058-41f7-b134-f00b0a7bb965 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.658707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591c00-5058-41f7-b134-f00b0a7bb965 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.658982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2591c00-5058-41f7-b134-f00b0a7bb965 None admin] PATH_INFO: `/v3/groups/4a90681ec46d435e8d13bff2ac87031c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.660246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2591c00-5058-41f7-b134-f00b0a7bb965 None admin] RBAC: Authorizing `identity:get_group(group_id=4a90681ec46d435e8d13bff2ac87031c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.661761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2591c00-5058-41f7-b134-f00b0a7bb965 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.663641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 224/454] 10.222.0.43 () {56 vars in 1180 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/groups/4a90681ec46d435e8d13bff2ac87031c => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:56.669962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:56.789456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f5d8b-11f2-48d3-849b-286c51673c42 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:56.826472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:56.827733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:56.828265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:56.828791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:56.830414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] RBAC: Adding query filter params (group.id=4a90681ec46d435e8d13bff2ac87031c, role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=58260e912d5d42f7b5b53106290e3849) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:28:56.830842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:56.832308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebaf790-2cb1-47ee-9df1-3caa83d0a4ef None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:56.838478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 231/455] 10.222.0.43 () {56 vars in 1376 bytes} [Fri Jun 24 08:28:56 2022] GET /identity/v3/role_assignments?group.id=4a90681ec46d435e8d13bff2ac87031c&scope.project.id=58260e912d5d42f7b5b53106290e3849&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:28:59.461328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabb7b4-7be9-49e8-88da-fe4565c829cf None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.461821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabb7b4-7be9-49e8-88da-fe4565c829cf None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.461821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabb7b4-7be9-49e8-88da-fe4565c829cf None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.463446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 225/456] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:59.468658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fce33-c45e-4dae-bd2d-ec047036b606 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.469051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fce33-c45e-4dae-bd2d-ec047036b606 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.469216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437fce33-c45e-4dae-bd2d-ec047036b606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.492251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-437fce33-c45e-4dae-bd2d-ec047036b606 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:59.493786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-437fce33-c45e-4dae-bd2d-ec047036b606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:59.549081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:28:59.585803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 232/457] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:28:59.634040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabb7b4-7be9-49e8-88da-fe4565c829cf None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:59.664554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2cdc-85b4-4b62-966b-aac394e25134 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.665166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2cdc-85b4-4b62-966b-aac394e25134 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.665669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c2cdc-85b4-4b62-966b-aac394e25134 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.667532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 233/458] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:28:59.678221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fb6ac-ae81-403b-9639-91c879736e11 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.678771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fb6ac-ae81-403b-9639-91c879736e11 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.679348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fb6ac-ae81-403b-9639-91c879736e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.707951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2c2fb6ac-ae81-403b-9639-91c879736e11 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:59.709118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2c2fb6ac-ae81-403b-9639-91c879736e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:59.799818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabb7b4-7be9-49e8-88da-fe4565c829cf None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:28:59.818405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 234/459] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:28:59.820093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14761-25be-4732-b7fb-979bb6947b58 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.820608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14761-25be-4732-b7fb-979bb6947b58 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.821301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14761-25be-4732-b7fb-979bb6947b58 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.827859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 235/460] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:28:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:28:59.829487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.829997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.830533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.852024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:28:59.852841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:28:59.853123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:28:59.853398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:28:59.859871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:28:59.861864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:28:59.879993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:28:59.882364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:28:59.919605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:28:59.920581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:28:59.921443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:28:59.921443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:28:59.921443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:28:59.921443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:28:59.937561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 226/461] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:28:59.938832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:28:59.978337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.031448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 236/462] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:28:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:00.032644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.114097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.193596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032d3f8-1b51-4bd5-b159-93f5a89c99c6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.233848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.234966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.235501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.236148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.237626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:00.238133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.239893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.257927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 237/463] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:00.259100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.265555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993dd169-d9b0-49a4-b99f-e341b5025f68 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.319081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.319937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.320218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.320490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.331386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.332850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:00.350151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:00.351116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:00.352614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:00.352614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:00.352614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:00.352614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:00.352614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 227/464] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:28:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 413 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:00.352614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.374839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.383864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b14171-6d9b-4b62-82b3-f0c08d056c24 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.421762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.425904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.426711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.427002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.427310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.437992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.439487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-476744aa-d66e-43cb-b3a7-b85b1e41e93d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:00.458192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:00.459072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:00.460232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:00.460232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:00.460232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:00.460232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:00.460456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7cf327-c23b-48b2-b5b4-085a6bf93c05 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.460529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 238/465] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:00.461063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f136c1c-8655-45f4-9fba-7c8935cdcbc0 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.461352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f136c1c-8655-45f4-9fba-7c8935cdcbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.461633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f136c1c-8655-45f4-9fba-7c8935cdcbc0 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.462890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 239/466] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:00.463604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f136c1c-8655-45f4-9fba-7c8935cdcbc0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.497464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.498241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.498438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.498693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.499429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.500871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.521908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 228/467] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:00 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:00.522738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.590404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f136c1c-8655-45f4-9fba-7c8935cdcbc0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.635670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.636478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.636863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.637315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.638197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:00.638618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.640223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231b7e5-29d6-4c1e-883a-167163676ae6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.659437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 240/468] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:00.659437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.659437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.659437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.669384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284bdf1b-e1c3-4c5a-a761-406aec7696cc None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.676829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:00.678256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:00.680861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.709889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.723937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.724473 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.725313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.725313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.727085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:00.727085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.728725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.750599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 229/469] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:00.751231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.770322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 241/470] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:00.771494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.862390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d6f9f-a04c-489a-b5eb-89291406dac5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.879966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68baddd-c756-4e55-86bc-0f6b9184d213 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:00.902036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.902639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.902921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.903196 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.903945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.905605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.915448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e59570-f1c0-4603-8c55-92a10670edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:00.916117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59570-f1c0-4603-8c55-92a10670edcb None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.916398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59570-f1c0-4603-8c55-92a10670edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.916834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e59570-f1c0-4603-8c55-92a10670edcb None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.917401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e59570-f1c0-4603-8c55-92a10670edcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:00.918812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e59570-f1c0-4603-8c55-92a10670edcb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:00.925987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 230/471] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:00 2022] POST /identity/v3/users => generated 248 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:00.927340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:00.936181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:00.959024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 242/472] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:00 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:00.960492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d55904-bdf5-4f4d-b0d2-88ba475f50f6 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.961088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d55904-bdf5-4f4d-b0d2-88ba475f50f6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.961749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d55904-bdf5-4f4d-b0d2-88ba475f50f6 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:00.964564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 243/473] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:00.974965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:00.977317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:00.977826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.014434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.019961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:01.023462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:01.126021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631cfd9-c43a-4d62-8662-513726d71c1b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.169938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.170668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.171396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.171396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.175279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 244/474] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:01.183614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.185086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.203090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:01.208315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:01.209193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:01.210090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:01.210090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:01.210090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:01.210090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:01.210090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 231/475] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:01.218685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.319891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.334085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9fe482-85b6-42a8-bd79-72b37036d157 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.352027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9524b3-4578-471f-86a4-ea68849c1faa None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.386456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.387997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.388651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.389174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.390225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:01.390677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.392882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.396086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.397117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.397579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.398114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.398593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.401947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.401947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.409253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.sql.core [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 08:29:01.409253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 08:29:01.409253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [parameters: {'id': 'fc48d942c3114200912af0ffbc81826d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 08:29:01.409253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 08:29:01.410305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 232/476] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:01.415918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result.read() Jun 24 08:29:01.440326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return ref.to_dict() Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:29:01.441278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application yield resource Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise self.value Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application session.commit() Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 08:29:01.442217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.session.flush() Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application compat.raise_( Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 08:29:01.443094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:29:01.444026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result.read() Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:29:01.444871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [parameters: {'id': 'fc48d942c3114200912af0ffbc81826d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:01.445679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 24 08:29:01.446503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:01.454314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 245/477] 10.222.0.43 () {60 vars in 1131 bytes} [Fri Jun 24 08:29:01 2022] POST /identity/v3/roles => generated 144 bytes in 252 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:01.463880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.587270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff487b3-5d33-4e15-98e9-69579aee1fdc None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.653426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.654574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.655975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.656426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.658131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.661447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.671163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21311bb0-1909-4f2b-8e39-2781414dcf28 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.706559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 233/478] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:01 2022] POST /identity/v3/users => generated 249 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:29:01.726471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d66e4-fdf4-4246-b794-0df5803a9dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.727567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d66e4-fdf4-4246-b794-0df5803a9dfb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.727988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d66e4-fdf4-4246-b794-0df5803a9dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.728398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d66e4-fdf4-4246-b794-0df5803a9dfb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.742004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0d66e4-fdf4-4246-b794-0df5803a9dfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-2c0d66e4-fdf4-4246-b794-0df5803a9dfb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:01.750572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:01.751749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 24 08:29:01.753278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:01.755611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 246/479] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:01.761425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:01.774711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:01.892189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76d4ae-38e5-456c-b634-cf70bed84fd7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:01.936881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:01.937578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:01.937858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:01.938133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:01.938913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:01.939241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:01.941343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa41d2-f263-438e-afe6-220dcdc94d5d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:01.951286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 234/480] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:03.009951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40095c3f-1552-4a5a-a75e-402cfc269615 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.010849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40095c3f-1552-4a5a-a75e-402cfc269615 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.011310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40095c3f-1552-4a5a-a75e-402cfc269615 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.012829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 247/481] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:03.023292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040654-e67c-4404-8f08-f62c1e1154f2 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.024017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040654-e67c-4404-8f08-f62c1e1154f2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.024132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd040654-e67c-4404-8f08-f62c1e1154f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.055105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bd040654-e67c-4404-8f08-f62c1e1154f2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:03.056322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bd040654-e67c-4404-8f08-f62c1e1154f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:03.148219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:03.157686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 235/482] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:29:03.190295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40095c3f-1552-4a5a-a75e-402cfc269615 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:03.262444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:03.336315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40095c3f-1552-4a5a-a75e-402cfc269615 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:03.363352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2940-6642-4363-be72-6ddfd2129c9f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.363660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2940-6642-4363-be72-6ddfd2129c9f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.363916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f2940-6642-4363-be72-6ddfd2129c9f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.365722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 236/483] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:03.370710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.371000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.371316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.389242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:03.390438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:03.396636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:03.397652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.398107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.398606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.418121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:03.426952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:03.427890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:03.428701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:03.436947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 248/484] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:03.443338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:03.467331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 237/485] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:03.487352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:03.603969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3307f7-ceb8-4ca6-bcc7-12c65b9e6cc3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:03.639832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:03.645858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:03.646729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.647018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.647329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.660649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:03.668852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:03.669762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:03.670593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:03.673324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324fb936-f07c-4f06-8501-58a3767f4ff8 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:03.675555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 238/486] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:03.677850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:03.690188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:03.749075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:03.749747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.750031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.750318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.751071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:03.751409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:03.752810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958c6b83-0754-4787-a510-e2698e86db07 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:03.771569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 249/487] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:03.773490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8a7be-9433-4cbf-ab03-6c8caedff296 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.774029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8a7be-9433-4cbf-ab03-6c8caedff296 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.774319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8a7be-9433-4cbf-ab03-6c8caedff296 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.775937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 250/488] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:03.776640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8a7be-9433-4cbf-ab03-6c8caedff296 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:03.819014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19320fa6-7050-471b-9659-902a0096f0bc None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:03.853964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:03.855378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.855378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.855378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.856109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:03.856460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:03.858004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6076de-ee25-4f37-a863-32f0f6cd6a85 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:03.868257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 239/489] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:03.869590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.869923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.870206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.888179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:03.889374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:03.906766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8a7be-9433-4cbf-ab03-6c8caedff296 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:03.953071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:03.954766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:03.955770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:03.956058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:03.956336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:03.971280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:03.973374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:03.983487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 240/490] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:03.984588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:03.992520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:03.994193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:03.996027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:03.996027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:03.996027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:03.996027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:03.996027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 251/491] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:04.004290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.038466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.115977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68b9d9-30bf-414a-9792-5a0081cd12a8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.143526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e754328-7e5d-4401-915f-8af8d8bc060f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.154480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.155310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.155600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.155878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.170048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.171825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.185745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.195234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.195234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.195234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.195234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:04.195234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.195234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:04.195682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:04.196467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:04.197339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:04.197339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:04.197339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:04.197339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:04.197339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 241/492] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:04.197339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.204518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 252/493] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:04.205969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.245647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.302336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.323618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12322bf9-3fe1-44d4-a6e2-08ee54f2db35 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.335459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca32950-56a5-4805-928b-1c0117a2cd35 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.373472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.374053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.374338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.374614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.375369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:04.375678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.377088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.377883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.379025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.379536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.380027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.390654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 253/494] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.391621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.392446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:04.401585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:04.402537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:04.403395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:04.409778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 242/495] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:04.409778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.499532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebc52b-9012-48a5-8976-8ddcdc9e48a7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.505376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.513215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f3821-e928-4f2c-b488-37c5ebf2cec1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.535872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.536649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.536929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.537199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.544931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.546377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.549676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.550471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.550759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.551038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.558687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.560172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ced36fdf-d3d3-4a19-b12f-e5e4e86532ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:04.561239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:04.562200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:04.563116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 254/496] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:04.563997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df34844-0e8e-4f8d-b080-e83f6d5713c8 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.563997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df34844-0e8e-4f8d-b080-e83f6d5713c8 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.564171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df34844-0e8e-4f8d-b080-e83f6d5713c8 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.566109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 255/497] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:04.566602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df34844-0e8e-4f8d-b080-e83f6d5713c8 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.569605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-4abe2a24-270e-4c40-9398-092020702ed8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:04.582230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:04.583349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:04.584303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 243/498] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:04.584882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55eab1-5b56-4a03-b63e-b14c7a8f3769 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.585112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55eab1-5b56-4a03-b63e-b14c7a8f3769 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.585338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de55eab1-5b56-4a03-b63e-b14c7a8f3769 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.586665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 244/499] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:04.588005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55eab1-5b56-4a03-b63e-b14c7a8f3769 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.669548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df34844-0e8e-4f8d-b080-e83f6d5713c8 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.687725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de55eab1-5b56-4a03-b63e-b14c7a8f3769 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.702496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.703162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.703479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.703760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.704508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:04.704817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.706732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad987f7-edb3-4241-bbdf-c5b652394ead None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.714663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 256/500] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.715782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abce45-9ca5-4564-979d-3512dc7b5c4e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.716068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abce45-9ca5-4564-979d-3512dc7b5c4e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.716349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73abce45-9ca5-4564-979d-3512dc7b5c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.722478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.723142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.723481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.723761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.724602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:04.724912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.726355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.733041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-73abce45-9ca5-4564-979d-3512dc7b5c4e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:04.734232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-73abce45-9ca5-4564-979d-3512dc7b5c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:04.763042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 245/501] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.763898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.778786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.793168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:04.801267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 257/502] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:04.802515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.802801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.803077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.819916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:04.821087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:04.873643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426500b2-6fad-41d3-b6f6-f8f0dd588a4e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:04.887627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 258/503] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:04.888531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.919649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:04.920333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:04.920617 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:04.920900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:04.921742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:04.922053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:04.923520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:04.945713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 246/504] 10.222.0.43 () {56 vars in 1137 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:04.945713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:04.990597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69497de5-52ae-414e-bae2-d6aeca3b1c39 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.014409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.015348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.026899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.027720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.028003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.028278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.038139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.039637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.045080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088a8815-9e83-407a-9677-467008dee8b7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:05.058016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:05.058919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:05.065126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:05.065126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:05.065126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:05.065126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:05.065126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 259/505] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:05.065126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.080512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.081161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.081450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.081731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.147723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.149464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.157270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964a820-ad4d-4e11-bed4-122cc334fd9e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.192857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.193883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.194163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.194442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.204385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:05.211997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:05.212925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:05.213760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 260/506] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:05.214580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.226795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.248654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.260563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 247/507] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:05.261726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.322526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fcbc9e-c6e3-43cc-8b14-2dc832e7d9ed None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.355166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.355977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.356258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.356532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.362711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9f863-4272-4193-8c52-314218f7f22b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.367933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.369461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:05.386799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:05.387830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:05.388726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:05.388726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:05.388726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:05.388726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:05.388726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 261/508] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:05.389000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.401175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.401824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.402104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.402378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.440363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.471393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.472885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.487627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.500436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd8fad-fb93-414e-b9f0-cd8c2f7c37c8 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.534866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.535558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.535849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.536126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.537137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:05.537445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.538855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.551258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 262/509] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:05.552058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.573590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 248/510] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:05.574769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.645369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.653234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e9f68-471a-4fe7-b782-5ecb9d96abcf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.676622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fea5a-3e6f-44eb-9bc1-479c5ecf809f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.687568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.688234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.688516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.688799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.690131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:05.690488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.691842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.699911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 263/511] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:05.700613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.712446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.715106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.716507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.716507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.716695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.726549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.728048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:05.746516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:05.747581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:05.748677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:05.748677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:05.748677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:05.748677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:05.748677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 249/512] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:05.748677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.802923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbc1a60-633b-449d-9989-0fa164b2d650 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.835953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.836616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.836901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.837180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.837917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:05.838225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.839694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.845704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408bab07-f30f-4308-8c44-d0310eeedd3b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.851117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 264/513] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:05.851961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.872053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.884857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.885654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.885938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.886220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:05.893274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:05.894726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:05.909164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:05.910084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:05.911015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 250/514] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:05.911644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:05.928172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:05.959666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a94a04-7e72-4ecb-9685-de76ac10802f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:05.994612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:05.995432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:05.995717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:05.995996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.005872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.007431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.008896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b4339c-a2d9-42e4-b2b8-25cb154baa94 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:06.025054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:06.025906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:06.026778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:06.026778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:06.026778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:06.026778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:06.027413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 265/515] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:06.028155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.043165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.043981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.044265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.044540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.055197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.056711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:06.073753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:06.074623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:06.075746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:06.075746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:06.075746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:06.075746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:06.075746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 251/516] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:06.076032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.108982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.131165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2159cb68-7269-49a6-a1a3-56da14ccdecf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.153676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.168179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.168852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.169142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.169429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.170159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:06.170474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.171890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.177783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cdd392-a139-4d6a-a98c-c681a6065dd7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.186082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 266/517] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:06.187049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.215769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.216383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.216667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.216948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.217648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.219094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.239479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 252/518] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:06 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:06.240281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.292126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9d89ce-279f-43de-80df-30732a3deb04 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.331228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.331918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.332201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.332477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.333326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:06.333632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.335095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.364375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.379898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 267/519] 10.222.0.43 () {56 vars in 1137 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:06.379898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.380148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33f942-650d-4b03-b528-e5a35fecf3de None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.419833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.420499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.421476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.421476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.421859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:06.422178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.424186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.428667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.438138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 253/520] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:06.438987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.494545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e4a0c-13d2-4499-b4b7-36cd54df8084 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.530211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.530883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.531173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.531476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.532206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:06.532514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.534440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.545105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8c5460-c3d2-4001-a32d-9eaa5c6857ea None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.545746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 268/521] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:06.546539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.586571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.587678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.588246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.588525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.596249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.597807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:06.612698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:06.613580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:06.614461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 254/522] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:06.614936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.627775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.651706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.654715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ea636c-fd35-4a53-862e-12e8fc2c65bf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.688930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.690104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.690386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.690659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.723823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7bbed9-4376-431c-a1f7-72a6f36f7a6c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.759662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.760374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.760813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.761090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.763934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.765421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.769083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.770530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:06.785091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:06.785993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 255/523] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:06.791975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.868031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.874875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:06.883726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 269/524] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:06.884544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.904060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0957e4db-4547-4812-979a-a47f05766b70 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:06.936968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:06.937842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:06.938125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:06.938402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:06.945982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:06.947456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:06.962147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:06.963330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 256/525] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:06.968894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:06.996614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcf2003-17fa-415a-b961-35bd4d0a85e7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.036354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.037218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.037510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.037792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.039876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.040205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.041667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.060745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 270/526] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.061555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.078528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c54108-9209-44bc-a46e-7c701723d99f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.107901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.115841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.117013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.117471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.117919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.118885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.119785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.121468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.125657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.145150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 257/527] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.145150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.174680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505ab2c0-317d-4577-8e7d-9d63163045d7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.210697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.211519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.211798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.212067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.222217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.223791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:07.241461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:07.242421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:07.243308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:07.243308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:07.243308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:07.243308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:07.243308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 271/528] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:07.243604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.259183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f46e6d-2fac-4e52-aecb-73eb59e7c6b2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.296316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.296911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.297197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.297474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.298438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.299416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.300857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.319720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 258/529] 10.222.0.43 () {56 vars in 1139 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.324710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.349569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6337423-c953-4ca0-8455-e26fea8f1579 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.363147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.378724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.390629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.391570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.391842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.392139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.393019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=b502d4020b494807b09e803df0b2a301) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.393359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.394803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.411249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 272/530] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/role_assignments?user.id=b502d4020b494807b09e803df0b2a301&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.411249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.448216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cffd5c2-e26c-451a-9ded-78504f43bae3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.498118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.498893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.499541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.499888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.501092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=f3229f7fdd344dda97a47709c93ce2e2) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.501481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.504104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.515433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 259/531] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/role_assignments?user.id=f3229f7fdd344dda97a47709c93ce2e2&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.516542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.567989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fcf74-635c-42f5-9660-c906cf64ff53 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.637363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.638291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.638724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.639170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.640061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:07.640514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.642039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.669072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3024f-55db-450c-b51b-f910e9af6c76 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.669864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 273/532] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:07.675727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.685423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.711827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.711827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.711827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.711827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.747959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.817182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.818842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.867693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a04502-8656-4cd5-aed2-6bb769ba6524 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:07.932969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:07.934015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:07.934460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:07.934877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:07.935001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 260/533] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:07.940012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:07.948046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:07.953869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:07.969472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:07.976087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:07.977168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:07.978223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:07.990073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 274/534] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:07.996418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.136418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe94d5-3427-4cf7-a594-37a56e4a5228 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.174363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:08.186347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c3decf-6a6d-49ed-aa06-25b4ad43dad7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.209343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.210313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.211022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.211329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.232054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:08.233548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:08.250245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.250926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.251232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.251506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.252842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:08.253153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:08.254593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:08.271857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:08.273647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:08.276023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:08.276023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:08.276023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:08.276023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:08.276023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 261/535] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:08.281789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.287834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 275/536] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:08.293534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.387449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:08.425922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99715e54-ce46-4a2b-b000-54f1344f3ac9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.428958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2f136-26da-4bb4-91f3-69fdf15c4eef None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.460232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.460910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.461171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.461528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.462398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=ffea41aca5bd4fd49e2c5f74ac317814) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:08.462714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:08.464586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400da6fd-fde2-4e0a-bc53-c9eedd727353 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:08.466186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.467168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.467784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.468262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.469189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:08.469712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:08.471527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 276/537] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:08 2022] GET /identity/v3/role_assignments?user.id=ffea41aca5bd4fd49e2c5f74ac317814&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:08.472102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:08.472812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150cce-783c-4e3d-b5ec-d1536d099eef None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.473098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150cce-783c-4e3d-b5ec-d1536d099eef None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.473375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d150cce-783c-4e3d-b5ec-d1536d099eef None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.474923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 277/538] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:08.488806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.489378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.489850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.491630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 262/539] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:08.496782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.526073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:08.527610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:08.557141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:08.611726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4c3d3-bcbe-478e-9628-cf1a5d8717a5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.652719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:08.661114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 278/540] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:08.663913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.664821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.665225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.665602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.672951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:08.674575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:08.689149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:08.690208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:08.691090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:08.691541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 263/541] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:08.700826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.712312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.854170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442a4c3-abaf-4fd8-9e85-edfc3f464673 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.901659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71addbe-3801-421b-bbe4-fd853a16a3cc None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:08.915363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.915948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.916229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.916503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.917301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:08.917612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:08.919055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:08.939815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 279/542] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:08.945539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:08.968651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:08.969458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:08.969741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:08.970014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:08.971521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:08.990964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:09.005110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:09.006031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:09.006868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 264/543] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:09.024984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:09.099560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:09.125479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ef0a3b-8d0d-4bdc-8994-311351312a4e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:09.186445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:09.187128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.187422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.187701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.188819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=95d010330c9c44ddb09dbd8c501f3087) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:09.189138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:09.191731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:09.196630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595ea0e-148a-4c08-b148-9907ed86c092 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:09.198202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 280/544] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:08 2022] GET /identity/v3/role_assignments?user.id=95d010330c9c44ddb09dbd8c501f3087&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:09.274163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:09.274988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.275186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.275365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.276141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:09.276448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:09.277868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:09.291921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 265/545] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:09.297474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:09.409485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:09.443970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc270-8539-4a85-84b3-34ab7202ddab None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:09.485375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:09.486496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.486917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.487362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.501964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:09.503673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:09.525177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:09.526253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:09.527361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:09.527361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:09.527361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:09.527361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:09.529240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 281/546] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:09.535305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:09.651300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2215ad0-ba8a-46ab-af8c-f1b4788d910d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:09.659385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:09.701405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:09.702035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.702319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.702592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.703340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:09.703646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:09.705619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:09.721916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 266/547] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:09.727540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:09.843259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804d8e00-d264-4ec7-bd08-60e9876019cc None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:09.869832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:09.879739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:09.880550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.880833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.881108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.889078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:09.890616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d8e03f8b-e62e-4edd-9eb8-b1c46f437d9f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:09.915358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:09.916406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:09.917799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:09.921107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 282/548] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:09.922092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:09.951725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5481540-16bc-43c1-ba56-a066b448937a None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.952002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5481540-16bc-43c1-ba56-a066b448937a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.952201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5481540-16bc-43c1-ba56-a066b448937a None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.953849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 283/549] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:09.963731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8d0be-976e-4491-9501-cad9b9c67221 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:09.964067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8d0be-976e-4491-9501-cad9b9c67221 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:09.964296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc8d0be-976e-4491-9501-cad9b9c67221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:09.983591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-afc8d0be-976e-4491-9501-cad9b9c67221 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:09.984776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-afc8d0be-976e-4491-9501-cad9b9c67221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:10.061560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 284/550] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:10.063244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65d139-d87e-401d-baae-f32f894d04bb None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.063657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65d139-d87e-401d-baae-f32f894d04bb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.063937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65d139-d87e-401d-baae-f32f894d04bb None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.064718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31328925-64d5-4b90-b2ea-46205fff4367 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.065844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 285/551] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:10.071421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.071713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.071997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.090188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:10.091497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:10.102140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.102927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.103423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.104021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.104954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:10.105359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.106882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.124982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 267/552] 10.222.0.43 () {56 vars in 1135 bytes} [Fri Jun 24 08:29:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:10.125893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.160846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.188111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 286/553] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:10.188111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.247959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.263534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f99a2e-4bef-485e-a58c-4c7dd6fd59c0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.299701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.300526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.300927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.301208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.308321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636200f2-f9c6-4412-9c0a-a4cd05ddd963 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.319774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:10.327932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:10.328989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:10.329842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 268/554] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:10.330587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.349070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.350166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.350670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.351241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.420227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.425532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.427283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.437879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fe2292-7103-4f9e-8521-5e59f50fa099 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.474624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.475513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.475749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.476026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.478662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.490159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:10.498534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:10.499695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:10.500533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:10.503286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 269/555] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:10.504571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.552118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 287/556] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:10.553426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.648048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4bf95c-df76-494c-996c-52d00ebd60b3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.675750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244fd134-c207-43eb-b873-d7ac131d0271 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.694167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.695325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.695681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.696033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.696863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:10.697246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.699269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.712405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 270/557] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:10.713324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.714699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.715604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.716231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.716703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.717624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:10.718132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.719706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.723612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.733786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 288/558] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:10.735937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.748757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:10.825651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a8647-fac5-43e0-b62d-76c42a752383 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.850863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a814257e-6eaf-47f9-9f90-eae8470f9f22 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:10.863131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.864010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.864298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.864582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.875123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.876613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.888770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:10.889559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.889844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.890121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:10.894366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:10.895242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:10.896165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:10.896165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:10.896165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:10.896165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:10.896165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 271/559] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:10.896438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.905835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:10.907328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-9b90b800-2638-40a1-afd6-3a77110ccdee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:10.926252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:10.927308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:10.928184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:10.928184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:10.928184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:10.928184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:10.928184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 289/560] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:10.928954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1f529-94be-4a3e-b325-c824623dffce None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:10.929067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1f529-94be-4a3e-b325-c824623dffce None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:10.929496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1f529-94be-4a3e-b325-c824623dffce None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:10.934734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 290/561] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:10.934734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1f529-94be-4a3e-b325-c824623dffce None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:10.963577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.000967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.009432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c4110-7def-4fee-a153-37e353c4f64c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.045119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1f529-94be-4a3e-b325-c824623dffce None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.046626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.047824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.048170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.048451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.061598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.063317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.079512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.080212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.080494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.080772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.081497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:11.081804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:11.082418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:11.083400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:11.084318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:11.084318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:11.084318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:11.084318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:11.084541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a8273b-e72d-4d84-bebc-7baebbcc2858 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.087358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 272/562] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:11.088383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.100830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 291/563] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:11.102162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.102456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.102737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.120020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:11.121412 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:11.193085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a73d1-a9f9-4202-8191-65906dd78d84 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.197933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 292/564] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:11.198781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.207093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.232841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.233777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.234206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.234658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.235557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:11.236015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.238066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.250700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 273/565] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:11.251898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.259531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.318417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71276502-0f67-48ae-9154-fa51b9ce8b99 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.355048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.355745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.356065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.356346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.357090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:11.357397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.358780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.359657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa4144-46f9-4a90-85fb-63d10066766a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.376091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 293/566] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:11.376091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.396019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.397030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.397498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.397778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.404886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.406331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:11.420551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:11.421475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:11.422393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:11.426972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 274/567] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:11.427832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.458535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.491891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccf2d69-b041-46b5-aba0-68898f9070cd None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.503429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.527868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.528658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.528938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.529215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.532841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d3998-e661-439e-bbd0-02943633840e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.539736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:11.547552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:11.548594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:11.550085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 294/568] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:11.550795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.568855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.569646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.569932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.570212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.577502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.578941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:11.593075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:11.594213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 275/569] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:11.595387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.649940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e09c-2fcf-492d-af2c-96960b872e69 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.682319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.683107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.683495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.683776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.687089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.691175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.692635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.697023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f273a-a88a-41df-a613-ace203d42711 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-34b400c5-d7c4-4c75-8452-6c6b144bbe57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:11.706843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:11.707996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:11.709302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:11.712731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 295/570] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:11.713964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c859da3-3117-4734-884e-10e582cf8bd9 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.714260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c859da3-3117-4734-884e-10e582cf8bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.714544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c859da3-3117-4734-884e-10e582cf8bd9 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.715837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 296/571] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:11.716526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c859da3-3117-4734-884e-10e582cf8bd9 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.733497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.734158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.734440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.734717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.735533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:11.735838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.737275 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.746609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.749252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 276/572] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:11.750034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.824438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c859da3-3117-4734-884e-10e582cf8bd9 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.844654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc57d29-13ac-4fbd-a099-44e93f582bbd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.858515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.859332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.859614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.859977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.860729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:11.861031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.862407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.870160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 297/573] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:11.870931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:11.883451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:11.884116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.884399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.884674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.885514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:11.885826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:11.887292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d741a328-e301-484a-bc51-1845bf245e75 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:11.904837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 277/574] 10.222.0.43 () {56 vars in 1137 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:11.906018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:11.906309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:11.906588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:11.922432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:11.923682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:11.925591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.967434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:11.976816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff251f9-e9b5-4595-a442-18f59270fd62 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:11.990128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 278/575] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:11.990782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.012756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.013433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.013720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.014003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.014874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:12.015187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.016641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.034322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 298/576] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.035118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.087796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79980b8-7eb4-46b5-a3f6-0fe551f1311c None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.120302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.120952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.121232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.121507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.122347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=69edde6d461545069a9e98c661e5453e) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:12.122665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.124125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.129860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 279/577] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:11 2022] GET /identity/v3/role_assignments?user.id=69edde6d461545069a9e98c661e5453e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.130622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.131338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb17c9b-f0aa-4f9e-a62b-27d127526a97 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.133023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.181592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.182491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.182852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.183198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.195082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.196624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:12.216612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:12.222927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 299/578] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.224127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.259617 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f079d-2ea3-4f58-8828-013121dc99b7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.300219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.301022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.301303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.301577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.317337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.318818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.335051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3acab1-e244-485b-a0f5-70b6fb2e3e4d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:12.337106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:12.338027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:12.338948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:12.338948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:12.338948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:12.338948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:12.342956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 280/579] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:12.343827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.373571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.374494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.374846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.375129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.386740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:12.395111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:12.396208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:12.397064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 300/580] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:12.397762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.405791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.453799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf99b70-51a3-4371-b8ff-3c32605a3dba None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.463748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.490815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.491920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.492402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.492852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.505547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56691b7b-3d79-4c4b-945a-afde5b78a676 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.507602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.509195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:12.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:12.528685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:12.529754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:12.529754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:12.529754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:12.529754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:12.531274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 281/581] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:12.532168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.545657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.546324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.546608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.546888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.547641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:12.547952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.549357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.563495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 301/582] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.564464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.645883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa4184-ad7c-4a51-a2d9-d5a83238149b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.653366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.676865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5041e5f-1c11-41ff-8f3b-1577e1ad4981 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.682184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.682852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.683141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.683528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.684276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:12.684644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.686020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.702853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 282/583] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.702853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.705387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.718195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.718953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.719387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.719622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.720370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:12.721262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.722552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.735197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 302/584] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.735197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.811159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ac73b-d24d-43c4-83b4-c744c114243b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.847898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.848140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f19590-acaa-41f7-afbe-09b790bc7006 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:12.848561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.848841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.849112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.849951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:12.850848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.851750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.869328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 283/585] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:12.869328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.887858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:12.888650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:12.888928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:12.889201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:12.895813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.896734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:12.898130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:12.912840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:12.913799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:12.914772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 303/586] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:12.915909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:12.962754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:12.977812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307de345-659f-42cf-8fb2-b39fa0340214 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.013187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.013973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.014251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.014520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.018778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa94b9-ca1a-41f3-ac49-e5775d3d30d5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.021463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.022926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:13.038773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:13.039903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:13.041028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:13.044867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 284/587] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:13.044867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.056933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.057798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.058156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.058433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.068949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.070389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:13.087590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:13.088517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:13.089438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:13.089438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:13.089438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:13.089438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:13.089438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 304/588] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:13.089727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.137677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.156009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa389d26-3c26-432c-9316-2f4cb20dbab4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.194550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.195381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.195665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.195942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.200448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.204005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.206194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.207976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beff87e-2fe2-4790-8259-25c1742b0168 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:13.221615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:13.222591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 285/589] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:13.224054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.252363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.253086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.253537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.253647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.254460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:13.254727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.256186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.278841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 305/590] 10.222.0.43 () {56 vars in 1137 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:13.278841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.333915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8e524f-55a2-4f94-898e-21b48a0b4170 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.369006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.369832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.370182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.370525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.371409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:13.371783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.373291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.381768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feff805-cc25-4926-ab05-699328226d8b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.390172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 286/591] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:13.390172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.398840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.422836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.423876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.424486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.424836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.425629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:13.426007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.428327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.442016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 306/592] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:13.442884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.459256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.500550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00bf7f7-7858-477b-9953-137e3c67053a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.541678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.542432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.542788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.543137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.544049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:13.544423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.545902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.549452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb12cbc-2b1f-4354-b761-636d85572e85 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.557435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 287/593] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:13.558209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.589878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.590711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.590994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.591309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.602391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.603886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:13.621805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:13.622715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:13.623653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:13.623653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:13.623653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:13.623653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:13.623653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 307/594] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:13.623952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.637629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.675543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411bc339-d626-4b74-98c0-47efd86bf26b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.708025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.714575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.715623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.716001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.716373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/f3229f7fdd344dda97a47709c93ce2e2/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.733209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=f3229f7fdd344dda97a47709c93ce2e2, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.734792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.746409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba118c-c3c7-4f74-b252-97d57435c13a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.759241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 288/595] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:13 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/f3229f7fdd344dda97a47709c93ce2e2/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:13.759241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.792222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.793205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.793569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.793946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.803847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.805459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:13.821460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:13.822493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:29:13.823507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:13.827550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 308/596] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:13.828519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:13.894817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680b8044-9e98-4d65-a97d-288746274bcd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.917333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:13.938964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:13.939866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:13.940148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:13.940429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/b502d4020b494807b09e803df0b2a301/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:13.957697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=b502d4020b494807b09e803df0b2a301, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:13.959469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:13.975528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ab3209-a7e0-4358-96ca-811b8f69622e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:13.981062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 289/597] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:13 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/b502d4020b494807b09e803df0b2a301/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:13.982806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.030404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.050535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.051351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.051745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.052087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.052894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:14.053267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.055872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.078217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 309/598] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:14.084659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.186948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da0c5ab-9ca6-4615-bac3-3eb542918837 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.227339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.227942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.228225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.228502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.229345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:14.229657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.231107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2771c190-6504-49fc-897c-a5bd6be09d34 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.246654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8de77-cb7e-4c3b-8cbd-83a107603c71 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.250620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 290/599] 10.222.0.43 () {56 vars in 1139 bytes} [Fri Jun 24 08:29:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:14.252725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2105e-0e40-462b-8772-9b1f0a7964bc None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.253203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2105e-0e40-462b-8772-9b1f0a7964bc None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.253656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2105e-0e40-462b-8772-9b1f0a7964bc None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.255539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 291/600] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:14.260062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2105e-0e40-462b-8772-9b1f0a7964bc None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.288182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.291470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.292258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.292541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.292816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.302404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.303960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4e570fae-72fa-4bbb-80c8-0959cbf03d2f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:14.319785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:14.320742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:14.321644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 310/601] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:14.322604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.322893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.323171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.341556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:14.342752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:14.376506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2105e-0e40-462b-8772-9b1f0a7964bc None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.389858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.415788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.416947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.417407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.423230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 311/602] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:14.423230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.423744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.443357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.445340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:14.472325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:14.473284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:14.474198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:14.474198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:14.474198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:14.474198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:14.476481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 292/603] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:14.477635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.526950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b63c2-e310-404d-91c3-2a5f45d8b8a0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.558179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.568536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.569208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.569489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.569765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.570558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:14.571458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.573806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.589997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 312/604] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:14.591439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.592615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e69cc-47c6-4f38-9953-043dc8def72b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.634100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.636936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.637858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.638391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.638676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.654276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:14.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:14.666645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:14.667546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 293/605] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:14.668248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.716376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d13b0-c26b-4f07-849e-bb7b2eaddbb1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.755051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.755684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.755968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.756246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.756963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:14.757268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.758731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.771560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 313/606] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:14.772373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.805625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81afc763-168e-498e-a61e-10491ba1622b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.851769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.852792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.853185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.853533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/ffea41aca5bd4fd49e2c5f74ac317814/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.857319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.872597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=ffea41aca5bd4fd49e2c5f74ac317814, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.874403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.884233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:14.893653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3510b-f6ab-4f43-8e76-d0186fb070f5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:14.896243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 294/607] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:14 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/ffea41aca5bd4fd49e2c5f74ac317814/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:14.897252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:14.941856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:14.943058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:14.943508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:14.943933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:14.945363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:14.945842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:14.947655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:14.960127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 314/608] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:14.966037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.093216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e94275a-7a85-41f8-9b4e-330a2e69c327 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.137276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4f98b2-b14f-4072-b899-2fcfbe24c382 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.138959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.140682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.140973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.141258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.155948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.158396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.178310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.178310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.178310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.178744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:15.187234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:15.188129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:15.189041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:15.189041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:15.189041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:15.189041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:15.189041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 315/609] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:15.189375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.189375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.191669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.205759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:15.206971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:15.213324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 295/610] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:15.215509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.273378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.310408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22802600-b684-4b5e-80ad-845c7ec9d6a1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.348193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a9a46-17a5-416f-8b9c-25c6daf7d8fb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.376041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.376783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.377074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.377356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.378104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:15.378416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.379956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.387117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.387828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.388111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.389161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.390008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:15.390385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.391905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.407696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 316/612] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:15.408590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 296/612] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:15.413690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.413913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.529803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d821b55-e49e-48e8-bc9b-29b47b437ce5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.539354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.559575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:15.570295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.572113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.572622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.573100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/95d010330c9c44ddb09dbd8c501f3087/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.588951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=95d010330c9c44ddb09dbd8c501f3087, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.590697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.607136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed351cec-6f12-430b-8e6b-3d88f0ae273e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.623690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 297/613] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:15 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/95d010330c9c44ddb09dbd8c501f3087/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:29:15.644302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.645275 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.645563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.645900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.654744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.656233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:15.673336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:15.674404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:15.675421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 317/614] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:15.680470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.815878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4190eb1-7f00-41e5-a1f5-88a6a077708d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:15.861411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:15.862167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:15.862547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:15.862846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:15.863739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:15.864047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:15.865596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:15.885116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 298/615] 10.222.0.43 () {56 vars in 1135 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:15.890522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:15.994416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7487b54-cbcd-480f-a200-544187304210 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.024827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.031021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.031892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.032176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.032565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.043381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:16.044895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7777568a-9ddf-44c9-b662-3d13b9355d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:16.062740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:16.063695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:16.065145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:16.065145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:16.065145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:16.065145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:16.065145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 318/616] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:16.070240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.086975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.133151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e013ae9-5f0e-4033-9241-ded81ac9f34f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.133663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e013ae9-5f0e-4033-9241-ded81ac9f34f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.134087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e013ae9-5f0e-4033-9241-ded81ac9f34f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.135890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 319/617] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:16.142837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.143360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.143785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.164536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:16.166267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:16.229944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c156c761-80eb-4aa8-93cb-b7ce776525e6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.269496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.270413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.270847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.271300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.272290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:16.272811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:16.274643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:16.288559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 299/618] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:16.293915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.322782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 320/619] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:29:16.377511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.421558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.485383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304360cf-2dfd-4495-b5b6-9c6a2ec58f64 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.529216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.530017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.530524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.530633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.533734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0cf0-24ee-4e7d-a670-00dc32e04572 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.539528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:16.540996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:16.548301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-29d55724-26ee-4775-97f6-1b48524bb2c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:16.563950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:16.565081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:16.565991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:16.566437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 300/620] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:16.572147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e5a7-5437-4eb3-b89e-164cf65f836e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.572701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e5a7-5437-4eb3-b89e-164cf65f836e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.572701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e5a7-5437-4eb3-b89e-164cf65f836e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.573563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f2b96-4e3e-4671-ad86-d5f6e503bace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.574198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 301/621] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:16.574908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350e5a7-5437-4eb3-b89e-164cf65f836e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.578364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f2b96-4e3e-4671-ad86-d5f6e503bace None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.579244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f2b96-4e3e-4671-ad86-d5f6e503bace None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.579244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f2b96-4e3e-4671-ad86-d5f6e503bace None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.590334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f2b96-4e3e-4671-ad86-d5f6e503bace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-950f2b96-4e3e-4671-ad86-d5f6e503bace None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:16.598738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:16.599885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:16.600736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:16.605630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 321/622] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:16.606915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e73e44-5099-438b-bb24-530cca890734 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.607233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e73e44-5099-438b-bb24-530cca890734 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.607517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e73e44-5099-438b-bb24-530cca890734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.624698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-83e73e44-5099-438b-bb24-530cca890734 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:16.625879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-83e73e44-5099-438b-bb24-530cca890734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:16.707033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 322/623] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:16.708295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e73e44-5099-438b-bb24-530cca890734 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.718653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350e5a7-5437-4eb3-b89e-164cf65f836e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.775135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.776724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.777256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.777731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.779381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:16.780353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:16.782520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:16.785821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.809100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 302/624] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:16.809100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.853992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e73e44-5099-438b-bb24-530cca890734 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.855890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:16.891985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.893560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.894046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.894534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.895557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:16.896078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:16.897644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:16.906898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 323/625] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:16.908671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:16.917826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc321892-28de-48f9-99f2-06313b97ac7f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:16.953251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:16.954054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:16.954338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:16.954614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:16.966192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:16.974741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:16.975762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:16.976654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:16.980701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 303/626] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:16.981534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.021605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5c5d1d-cbfd-4a51-bc21-d8d9c4573ad3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.044633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:17.059777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.061026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.061504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.062126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/69edde6d461545069a9e98c661e5453e/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.082468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=69edde6d461545069a9e98c661e5453e, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.084913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.111362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d96eae-f852-402b-8624-a00b4c7afdfa None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.111685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 324/627] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:16 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/69edde6d461545069a9e98c661e5453e/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:17.115893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.137379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:17.218708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.219790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.220221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.220690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.247539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.249288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:17.281049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:17.282052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:17.283097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:17.283097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:17.283097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:17.283097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:17.286552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 304/628] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:17.296985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe394e-e2e9-444d-a797-89b64f9346c0 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.302372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.336507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.337227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.337472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.337753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.338850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:17.339161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.340726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.355362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 325/629] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:17.357781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.433706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1445aa-4834-40bd-abc9-f3be2e3c24fb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.443298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:17.476215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:17.485070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.486832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.487609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.488072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.489002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:17.489493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.491819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1681ac-bb64-4bd4-8074-7a3a3e4137d0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.502725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabc0124-61ac-4fb3-8f7c-57dd38025214 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.508463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 305/630] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:17.510077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00717549-dc9d-46db-9628-3425711458f3 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.511104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00717549-dc9d-46db-9628-3425711458f3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.511605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00717549-dc9d-46db-9628-3425711458f3 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.513472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 306/631] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:17.518484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00717549-dc9d-46db-9628-3425711458f3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.549784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.550582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.550876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.551153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.563079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.564851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-273aa57a-f59d-4437-bb5e-2a321091aae9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:17.583992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:17.584981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:17.585876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:17.585876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:17.585876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:17.585876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:17.589851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 326/632] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:17.591570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.591777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.592048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.610442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:17.611712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:17.656352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00717549-dc9d-46db-9628-3425711458f3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.691260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 327/633] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:17.691260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.716559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.717387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.717676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.717949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.730231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.731621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:17.746403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:17.747466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:17.748448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 307/634] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:17.749058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.777905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:17.779499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:17.831742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292e9ab5-3cbb-4f0a-9ecf-4bb5697fb921 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.867433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1879c887-bb24-4cc0-942a-ede5bb836338 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:17.871867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.872667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.873103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.873490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.874323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:17.874732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:17.876244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:17.894358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 328/635] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:17.895505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:17.910000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:17.911359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:17.911789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:17.912506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:17.925853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:17.936534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:17.937586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:17.938444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:17.946657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 308/636] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:17.946657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.014603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb184ed-2689-4d77-a9c4-923908d92d4d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.033173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.039789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.052334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.053008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.053290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.053568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.054433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:18.054746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.056201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.059129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70be040e-806a-459d-ab2b-fa21c9536ad4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.078875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 329/637] 10.222.0.43 () {56 vars in 1137 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:18.080575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.095698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.096488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.096774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.097060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.104631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.106067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-5bae4ab0-a110-4898-a8ba-51a4a23b0011 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:18.120917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:18.121976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 309/638] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:29:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c305f4e-a418-4554-a884-b81104f22d8b None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.128274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c305f4e-a418-4554-a884-b81104f22d8b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.129001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c305f4e-a418-4554-a884-b81104f22d8b None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.130033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 310/639] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:18.130812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c305f4e-a418-4554-a884-b81104f22d8b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.191234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aedca0-6c8e-4332-b43b-1f5e2671c0d1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.227880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.228552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.228838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.229118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.229868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:18.230177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.231564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.241136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 330/640] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:18.242020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.247056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c305f4e-a418-4554-a884-b81104f22d8b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.283132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.283724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.284014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.284292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.285042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.300485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.361571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.362574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdde76-95ca-4bf3-8d8c-931b4bb9df40 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.363015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.397864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.399426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.399426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.399426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.400172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:18.400478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.401897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c1a1b-b102-440a-a6d1-2e0eb125c07d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.424108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 331/641] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:18.424108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.424108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.424108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.437844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:18.439046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:18.480548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 311/642] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:18.481798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.515125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 332/643] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:18.516471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.544439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.555167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.603634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f6583-1700-4a8d-9a30-61d0e94a5c70 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.637780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b04713-2bee-406d-a5e8-b85861205857 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.641835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.642665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.642949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.643251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.654537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.656841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:18.676044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:18.677086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:18.678007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:18.678007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:18.678007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:18.678007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:18.678007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 312/644] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:18.678307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.692274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.692950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.693233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.693517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.766057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.767591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.784127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.788810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd274-93c8-47d4-956d-de2adc6f057d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.795842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:18.826993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:18.827831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:18.828117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:18.828603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:18.841982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:18.843812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:18.862327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:18.863344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:18.864428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:18.864428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:18.864428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:18.864428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:18.864428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 313/645] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:18.864720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.882516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 333/646] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:18.884331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:18.972889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75049aa6-4990-44dd-b746-9264045b9e1f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:18.996778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0c1945-db70-491c-ad39-8c84548e96af None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.008889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.009685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.009963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.010237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.021164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:19.025014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:19.029389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:19.030367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:19.031286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 314/647] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:19.032017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.041003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.041763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.042055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.042337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.043086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:19.043430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.044810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.050020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.058540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 334/648] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:19.059199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.150595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4c17a-2c85-4739-b48d-e144fcf4540d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.168159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8517b-65b5-48a5-92b2-0d068f4b934b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.193510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.194180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.194466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.194745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.195511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:19.195822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.197217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.209541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.209989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 315/649] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:19.210707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.214715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.215004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.215308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.227890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.229338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:19.247781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:19.248742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:19.249641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:19.249641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:19.249641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:19.249641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:19.253924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 335/650] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:19.254852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.283571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.295402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.315750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a56305-6a7d-4226-9d4e-e333f365a3fb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.348008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.348663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.348944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.349218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.349992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:19.350299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.351703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.354725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ca12e-a4a9-4a1b-9596-e25dfb9417a9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.359288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 316/651] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:19.360053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.389227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.390019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.390301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.390574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.398110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.399851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:19.413557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:19.414536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 336/652] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:19.415528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.459756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af655b7-005d-4db3-b445-383c2b58d4a7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.489552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.495107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.495958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.496238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.496514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.503119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.504588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.511143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.515089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba555557-16c1-4c48-99d6-5428cdb3e930 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-20fb7463-0796-49c4-b0c3-239a1909f0ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:19.518630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:19.519802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:19.520706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 317/653] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:19.521425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b858425-67d4-4083-ba1e-358718fcd764 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.521425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b858425-67d4-4083-ba1e-358718fcd764 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.521552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b858425-67d4-4083-ba1e-358718fcd764 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.522780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 318/654] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:19.523503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b858425-67d4-4083-ba1e-358718fcd764 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.551994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.552658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.552944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.553221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.553944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:19.554254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.555645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.567057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 337/655] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:19.567817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.630635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b858425-67d4-4083-ba1e-358718fcd764 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.662902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b9dfa-3c36-4c85-9469-9a1316523643 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.664208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.664999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.665278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.665559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.675870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.677312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:19.694226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:19.695350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:19.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:19.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:19.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:19.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:19.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 319/656] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:19.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.698740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.699426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.699803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.700138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.700975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:19.701282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.702698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e093bb-be39-4dcb-b9bc-941e46f69b49 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.723969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 338/657] 10.222.0.43 () {56 vars in 1137 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:19.723969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.723969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.723969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.737659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:19.738837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:19.744529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.749469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.799140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84a984-5837-4cb6-a474-b28891bda46e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.804831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 339/658] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:19.805645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.846982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.847605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.847890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.848169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.848964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:19.849278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.850714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.862012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 320/659] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:19.862794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.902721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235f5d6-8225-4ac8-a823-98d4278328a7 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.935707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:19.936503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:19.936787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:19.937063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:19.943677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:19.945115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:19.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:19.960030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:19.960926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 340/660] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:19.961592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:19.967740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53856511-f68b-4aa5-a963-199939a76eae None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:19.969590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:19.970093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.002066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.002719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.003001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.003306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.004029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:20.004333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.005697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.018245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 321/661] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:20.019024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.059899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008bce-2dbf-4673-923e-1a00a37181ce None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.093190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.093832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.094110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.094382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.117513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da0664f-8654-472b-bb05-87a70c992018 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.150978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.151779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.152065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.152347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.161408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.162579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:20.162866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:20.170290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:20.171261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:20.173972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:20.174845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.174931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 322/662] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:20.174931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.198859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.265928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 341/663] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:20.267056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.271915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d964fc5b-1a16-4922-9c7b-2ee056f061ae None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.305017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.305671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.305956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.306237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.307071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=f3229f7fdd344dda97a47709c93ce2e2) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:20.307414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.308853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.315701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 323/664] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/role_assignments?user.id=f3229f7fdd344dda97a47709c93ce2e2&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:20.316593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.397972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81a04c7-832f-432e-aa91-4f2ac440e5da None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.406592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.420462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.435820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37f6f37-9645-48a6-af74-8bb6661372ad None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.437642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.439126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.439126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.439126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.439736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:20.440045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.441480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.459074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 342/665] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:20.459074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.486811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.487710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.488001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.488324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.497024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.498488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:20.513768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:20.514920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 324/666] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:20.516497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.580656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53902ed5-fe65-43b2-b88a-9b547771d98e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.616879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.617556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.617839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.618116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.618905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:20.619239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.620632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.625383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32102536-602d-4609-bb3e-1a98c0b63d7b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.633179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 343/667] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:20.633179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.661985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.664559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.665393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.665681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.665980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.677369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.679106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.687931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:20.700005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:20.700983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:20.701926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:20.701926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:20.701926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:20.701926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:20.701926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 325/668] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:20.702231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.739489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac602f8c-97e1-46b0-b7e6-b16473ea46c8 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.774618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.775770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.776261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.776760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.777626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=b502d4020b494807b09e803df0b2a301) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:20.777973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.779957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.786683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 344/669] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/role_assignments?user.id=b502d4020b494807b09e803df0b2a301&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:20.787511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.821084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d600be8-a311-476d-b9f5-aebeb1eebef3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.875393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.877707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.878359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.878791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.879866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:20.880358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.882015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.905550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.910414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 326/670] 10.222.0.43 () {56 vars in 1139 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:20.911928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:20.917394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c066aa13-affd-491d-a590-fd1ca37f9f47 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:20.958972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:20.960108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:20.960559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:20.960995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:20.974214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:20.976684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:20.988911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:20.996342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:20.997425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:20.998555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:20.998555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:20.998555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:20.998555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:21.005066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 345/671] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:21.006079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.047105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24994c1b-96d1-48b2-9d20-0cd114c83e4f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.092871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.093548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.093832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.095287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.095287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:21.095287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.096569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.111709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 327/672] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.112381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.133524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e56f9c-9594-4cc9-91d9-f250d2a3562b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.172493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.173067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.173378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.173662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.190168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.225324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911fd242-98df-4b58-8c73-f9d3dcd613d8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.242881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.248308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.249956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.262415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.263304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.263661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.264054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.264904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:21.265286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.266755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.284645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 328/673] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.285583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.357885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 346/674] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:21.359033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.389336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f5844a-67c8-41c8-b35a-f17b4942f1d0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.413079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.425625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.426540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.426837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.427243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.434280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.435854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:21.450560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:21.451561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:21.452463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:21.456545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 329/675] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:21.457379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.471273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11758ab-d479-4e0e-90ed-53751f4cea9e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.498916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.511462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.512788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.513282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.513861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.522867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:21.541458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:21.542427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:21.543402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:21.545721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 347/676] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:21.546633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.576896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d8183-a2b9-4624-82ac-e14b7cb0e808 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.625566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.627661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.628484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.628950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.642503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.644586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:21.666656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:21.668054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:21.668955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:21.668955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:21.668955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:21.668955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:21.668955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 330/677] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:21.669902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.675368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16470233-5a6f-4c2f-be6d-4277e086350c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.678740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.719417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.720137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.720947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.721229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.722021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:21.722332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.724310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.743289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 348/678] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.744399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.789745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.805897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71e95af-4df1-4a51-9e46-8b2c9160701d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.843443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.844117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.844453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.844782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.845679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:21.846156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.847542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.861312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf4acc-03ab-4f45-ae8d-e7ee9945489f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:21.870878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 331/679] 10.222.0.43 () {56 vars in 1135 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.871694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.897451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:21.898459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:21.899254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:21.899734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:21.900659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:21.901174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:21.902734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:21.915179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 349/680] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:21.916346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:21.941819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:21.985846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb70320-4488-44d9-a0c0-da2a2e04f6e3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.019622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d1ba0a-40ad-4bb9-8d80-ba27adade7e9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.024951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.025822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.026260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.026729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.027902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=ffea41aca5bd4fd49e2c5f74ac317814) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:22.028392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.030133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.042155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 332/681] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/role_assignments?user.id=ffea41aca5bd4fd49e2c5f74ac317814&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:22.043472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.057494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.057494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.057494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.057494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.075050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:22.135258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.135693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.161308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:22.244201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 350/682] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:22.252997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.272797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f871b9-e721-4fe5-8edf-1cd3c5d4ec09 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.332818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.333292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.333581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.333861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.341448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.342912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:22.358018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:22.359186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:22.360380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:22.363204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 333/683] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:22.368861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.393495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7598041-fbd5-446a-88ac-83f9419ede0c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.429690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:22.444561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.445584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.446146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.446584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.450859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:22.458678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.461132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:22.481291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:22.482465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:22.483670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:22.483670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:22.483670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:22.483670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:22.484751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d834b-c727-47ef-bff8-04caf01803a9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.485722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 351/684] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:22.491759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.521978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.522567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.522846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.523336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.524246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:22.524549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.526123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03e411e-423a-4f86-8b87-54e78fbd9315 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.539869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 334/685] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:22.541445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d003fb-6aa6-4f0f-9997-16d81af591cc None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.541737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d003fb-6aa6-4f0f-9997-16d81af591cc None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.542015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d003fb-6aa6-4f0f-9997-16d81af591cc None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.547547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 335/686] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:22.549138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.549463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.549743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.568526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:22.570233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:22.612339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8554cfa-e4ad-4df1-9f49-f60af89023c3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.651428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.652186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.652461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.652811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.653520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:22.653835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.655221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.668020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 352/687] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:22.668963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 336/688] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:22.669417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.675305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.729913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:22.755262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:22.805247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668603bd-cba2-48d9-b941-d943797ea2ae None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.807028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643082ce-1c50-4e84-9bab-2c3d2eb55a26 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:22.844247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.844930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.845784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.846046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.846884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=95d010330c9c44ddb09dbd8c501f3087) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:22.847198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.848690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.849225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:22.850243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:22.850863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:22.851368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:22.855096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 353/689] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/role_assignments?user.id=95d010330c9c44ddb09dbd8c501f3087&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:22.855882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:22.860055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:22.861837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:22.880184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:22.881179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:22.882107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:22.885104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 337/690] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:22.891568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.061391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:23.066847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4883261-78c9-4dbd-9857-be8815faf873 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.081311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc53a5c-c5aa-4421-80fd-d7e7d3a71b32 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.110894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:23.116924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.117807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.118037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.118316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.118984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.120404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:23.128153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.129113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.129601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.130087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.131089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:23.131657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.133290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c452ad-116e-4fb2-aa5e-3cad85da4c1e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-e7856052-02e4-4150-a2e2-1079e203a1f0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:23.135018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:23.136140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:23.141820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 354/691] 10.222.0.43 () {56 vars in 1129 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:23.143249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362477f-5a81-42d2-9b54-d3aa1e75db9b None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.143249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362477f-5a81-42d2-9b54-d3aa1e75db9b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.143249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362477f-5a81-42d2-9b54-d3aa1e75db9b None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.143249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 355/692] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:23.143249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362477f-5a81-42d2-9b54-d3aa1e75db9b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.153279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 338/693] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:23.154621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.154851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.155132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.177847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:23.179434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:23.254184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 339/694] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:23.255479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.296649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3362477f-5a81-42d2-9b54-d3aa1e75db9b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.342090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.343510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.343999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.344462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.345882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:23.346368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.347975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:23.353212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 356/695] 10.222.0.43 () {56 vars in 1124 bytes} [Fri Jun 24 08:29:23 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:23.358623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.366335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c92f08e-6947-489f-abb1-93b2f0c40844 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.383712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:23.391800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:23.416280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.416944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.417232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.417512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.418611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=69edde6d461545069a9e98c661e5453e) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:23.418930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.420823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:23.435240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 340/696] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:23 2022] GET /identity/v3/role_assignments?user.id=69edde6d461545069a9e98c661e5453e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:23.435240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.512890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dba65b-745a-4f18-959c-1ce09b1ff51e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.563868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc094a15-521e-48f4-9d08-a379148674d4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.601464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.602618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.603142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.603674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.611657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.612352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.612635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.612915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.613647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:23.613957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.615404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:23.626454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 341/697] 10.222.0.43 () {56 vars in 1124 bytes} [Fri Jun 24 08:29:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:23.628593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:23.636529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:23.637595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:23.638447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 357/698] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:23.644824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.695258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:23.761648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0950d8e-cbdf-4e9e-8f3d-61293339af29 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.803428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.804123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.804402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.804674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.805971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:23.806294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.807707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:23.819467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 342/699] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:23.824750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:23.928196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:23.942862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7b8e4-7300-46a3-82ad-daf6d1b07335 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:23.978532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:23.979387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:23.979724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:23.980008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:23.992934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:23.994388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:24.012520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:24.013486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:24.014391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:24.014391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:24.014391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:24.014391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:24.014391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 358/700] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:24.020154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.155916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e99aa-64b5-40db-89db-259bb89b16d9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.157447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:24.205919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.206592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.206932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.207256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.207975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:24.208282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.209659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.222289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 343/701] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:24.227643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.365404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd78629-240a-4582-94ed-3f1bc7d1137c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.410856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.411684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.411971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.412254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.426199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.427682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.433732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-db3d0e78-5429-4631-a867-74730a316ac1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:24.448178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:24.449180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:24.450207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 359/702] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:24.455758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.577281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5da27-6f12-4b62-bed2-80232af4b11c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.614672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6bcb8-0dc2-4f21-890b-172c4aa781af None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.615243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6bcb8-0dc2-4f21-890b-172c4aa781af None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.615243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6bcb8-0dc2-4f21-890b-172c4aa781af None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.616740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 360/703] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:24.621850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.622377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.622882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.638476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.639590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.640127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.640607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.641689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:24.642732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.648396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:24.649774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:24.651169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:24.671932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 344/704] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:24.678496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.715646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:24.792077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 361/705] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 24 08:29:24.818725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:24.868017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e65f521-cc4a-41d3-b7f2-790098726d0b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:24.912141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:24.912804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:24.913089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:24.913368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:24.994481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:24.995985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.022629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.077985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9914d2-1312-473d-ba81-fb5d367135dd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.115545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 345/706] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:25.121120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.151578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.161776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.162620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.162903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.163377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.186279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.187787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:25.224751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:25.225901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:25.226851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:25.226851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:25.226851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:25.226851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:25.231368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 362/707] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 414 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:25.232355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.286175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96a5e0-cb10-4f58-abdb-1f98162603ab None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.310364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.335577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.336379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.336656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.336926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.354324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.355830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-31a9155c-7996-43f9-bb80-c299342fa5ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:25.396341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:25.397311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:25.398226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:25.398226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:25.398226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:25.398226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:25.399273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 346/708] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:25.402651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb1ced-ad71-4e1a-ae35-b5a125d7a252 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.402651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb1ced-ad71-4e1a-ae35-b5a125d7a252 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.402651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb1ced-ad71-4e1a-ae35-b5a125d7a252 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.406550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 347/709] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:25.407349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb1ced-ad71-4e1a-ae35-b5a125d7a252 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.431587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f13a10-a074-42c2-89d4-6e743b793981 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.480476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.481203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.481846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.482127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.483133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:25.483605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.485696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad47a98-2725-42cc-823e-d74f22b6f137 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.501786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 363/710] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:25.503718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.504218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.504720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.528514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:25.530053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:25.535271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb1ced-ad71-4e1a-ae35-b5a125d7a252 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.568920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.580506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.581480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.582034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.582320 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.583062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:25.583403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.584962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.600904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 348/711] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:25.601965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.607418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 364/712] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:25.608596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.648613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.727466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c149a6-6d14-4281-a8e5-f702ff26edc5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.743552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06f087-36d4-4f57-9557-f5b8d3c80864 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.762976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.763498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.763781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.764052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.764831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.766199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.789293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.790468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.790949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.791494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.792590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 349/713] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:29:25 2022] POST /identity/v3/users => generated 254 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:25.793560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.800799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.802881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:25.821089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:25.822173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:25.823157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:25.825920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 365/714] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:25.832139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:25.909620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.928339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fb6ae0-679d-4f70-bf9f-332f74c9b0b7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:25.971912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:25.972617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:25.974470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:25.974757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:25.975037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:25.975837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:25.977477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:25.995974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:25.997094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:25.997967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:26.003095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 350/715] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:25 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:26.004362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:26.074059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e216d74-fc58-4a8d-990b-a729190b235a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:26.119018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:26.119747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.120017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.120284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.121138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:26.121391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:26.122835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b095a25-1c04-48e0-92d5-b4c0668f2a81 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:26.136843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 366/716] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:26.137474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cbef94-7c4c-4fd9-b812-0e40af968da3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:26.138151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9a75e-0f65-48e6-9585-5e81f176614c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.138444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9a75e-0f65-48e6-9585-5e81f176614c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.138725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df9a75e-0f65-48e6-9585-5e81f176614c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.140266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 367/717] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:26.144261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9a75e-0f65-48e6-9585-5e81f176614c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:26.181936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:26.182619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.182906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.183183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.184004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:26.184312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:26.185694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dfa2cf-2206-4bd2-a3de-695d83acc7a4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:26.193831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 351/718] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:26 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:26.193831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.193831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.193831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.224779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:26.227015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:26.242599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:26.262856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:26.276958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df9a75e-0f65-48e6-9585-5e81f176614c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:26.313016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 352/719] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:26.314569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:26.319949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:26.320861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.321343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.321813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.322966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=b502d4020b494807b09e803df0b2a301) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:26.323504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:26.325304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d7e18-4089-478e-ac0e-1960c4fecbee None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:26.332442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 368/720] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:26 2022] GET /identity/v3/role_assignments?user.id=b502d4020b494807b09e803df0b2a301&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:26.334051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1933-53d6-440a-ac06-fda62cb94c62 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.334554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1933-53d6-440a-ac06-fda62cb94c62 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.335047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd1933-53d6-440a-ac06-fda62cb94c62 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.337004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 369/721] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:26.338217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1933-53d6-440a-ac06-fda62cb94c62 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:26.436700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b47a95-1d9b-4692-8be4-515768d94fc1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:26.477968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:26.479198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.479818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.480101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.481179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] RBAC: Adding query filter params (type=network) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:26.482000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:26.483718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb6c32-a118-4001-9393-b53455ff4bea None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:26.489821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 353/722] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:26 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:26.490693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd1933-53d6-440a-ac06-fda62cb94c62 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:26.491969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.492646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.494046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.530239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:26.532495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:26.537869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:26.541634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:26.542325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.542742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.543023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.544099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:26.546962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:26.571692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 370/723] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:26 2022] POST /identity/v3/services => generated 251 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:26.634304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 354/724] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:26 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:26.670902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:26.700742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:26.801402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4fdaac-58d1-4a10-bed5-e23835e912ab None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:26.838091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:26.839277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:26.839791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:26.840276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:26.851500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:26.859399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:26.860352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:26.861778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:26.864557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 371/725] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:26.871061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:27.014964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7752cc0a-de85-47a9-ad50-bf23d5102264 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:27.049681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:27.050350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.050633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.050909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.051689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:27.051995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:27.053368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:27.055595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:27.062383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 355/726] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:27.067743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:27.193381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:27.232829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2f6002-e9f5-4d96-9126-56c3173c9925 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:27.284880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:27.285661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.285948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.286222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.307277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:27.308726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:27.330747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:27.331729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:27.332672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:27.332672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:27.332672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:27.332672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:27.336419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 372/727] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:27.341895 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:27.528793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9875a6fc-c7e0-48eb-8c0a-f1a29d1951c7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:27.598273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:27.598992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.599300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.599584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.600316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:27.600623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:27.601986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:27.622427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 356/728] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:27.628038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:27.790090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8755d7c7-c1a6-481a-b71a-0330acc19bf7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:27.818371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:27.836503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:27.837322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:27.837607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:27.837887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:27.848226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:27.849753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:27.867700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:27.868678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:27.869675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 373/729] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:27.876988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:27.975383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:28.027412 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff2fd2-aeb6-47dd-a1eb-8146540b0236 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:28.073958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:28.074636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.074924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.075204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.076071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:28.076381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:28.077836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c759b-a7d0-404b-b853-9c74d64735b8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:28.106829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 357/730] 10.222.0.43 () {56 vars in 1135 bytes} [Fri Jun 24 08:29:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:28.112837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:28.282004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a57c43-ed39-46de-aced-2ddb915f09f6 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:28.330346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:28.331014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.331324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.331608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.407388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:28.425416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:28.426865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:28.530462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598333d-8ce5-42ed-a260-588270369145 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.531071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598333d-8ce5-42ed-a260-588270369145 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.531559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4598333d-8ce5-42ed-a260-588270369145 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.533865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 358/731] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:28.539609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebef848-d758-457f-98aa-3749b06315cd None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.539896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebef848-d758-457f-98aa-3749b06315cd None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.540180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebef848-d758-457f-98aa-3749b06315cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.570241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 374/732] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:28 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 24 08:29:28.571228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-cebef848-d758-457f-98aa-3749b06315cd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:28.572433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cebef848-d758-457f-98aa-3749b06315cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:28.575344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:28.675299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 359/733] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:28.719240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebef848-d758-457f-98aa-3749b06315cd None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:28.750991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:28.772548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141ed593-8dff-4469-b91e-8cbbb22e9bf5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:28.797766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:28.814438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:28.815270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.815605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.815885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.826977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:28.828456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-3e56e6c5-0c9a-41b5-a714-e0492c0704ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:28.848096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:28.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:28.849959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:28.849959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:28.849959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:28.849959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:28.849959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 375/734] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:28.854025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63caaa-1c01-490d-a577-6739366afaf2 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.854025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63caaa-1c01-490d-a577-6739366afaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.854025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c63caaa-1c01-490d-a577-6739366afaf2 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.854280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebef848-d758-457f-98aa-3749b06315cd None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:28.854357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 376/735] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:28.854754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63caaa-1c01-490d-a577-6739366afaf2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:28.903237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d552901-93b8-4c97-a1e9-cbb8ac957028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:28.904070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d552901-93b8-4c97-a1e9-cbb8ac957028 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.904357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d552901-93b8-4c97-a1e9-cbb8ac957028 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.904641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d552901-93b8-4c97-a1e9-cbb8ac957028 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.918847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d552901-93b8-4c97-a1e9-cbb8ac957028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-7d552901-93b8-4c97-a1e9-cbb8ac957028 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:28.930158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:28.931097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:28.932302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 360/736] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:28.933388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:28.933641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:28.933911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:28.951329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:28.952524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:28.976145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c63caaa-1c01-490d-a577-6739366afaf2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.010818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.011512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.011795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.012070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.012793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:29.013100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.014484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb21ef0c-4366-4942-994d-b1f1d7fc0633 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.022666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.026075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 377/737] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.026774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 361/738] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:28 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:29.027479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822678ca-cefe-4e0e-8e2f-5d56eaa663f4 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.027818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822678ca-cefe-4e0e-8e2f-5d56eaa663f4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.028145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822678ca-cefe-4e0e-8e2f-5d56eaa663f4 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.028664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.033636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 378/739] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:29.036590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822678ca-cefe-4e0e-8e2f-5d56eaa663f4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.071906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.143954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c842b19-2530-4140-b497-d4ae5d6f74e1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.147819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822678ca-cefe-4e0e-8e2f-5d56eaa663f4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.179533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.180632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.180632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.180807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.181491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:29.181917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.183569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f184fb-bff7-47e0-9649-ab0205c954e0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.192316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.197224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 362/740] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.197338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.197338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.197338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.197596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.197596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.197596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.205003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.206451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.214011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:29.215353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:29.221753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:29.222672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:29.223645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:29.227513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 379/741] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:29.229044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.288804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 363/742] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:29 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 24 08:29:29.289655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.303432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.326917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.349155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603950a5-72f4-4baa-98e6-ebf6d446b85d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.385383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.386272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.386574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.386856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.397388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:29.405242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:29.406220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:29.407069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:29.408772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c672a-7f75-4f99-b0cf-f6f6d1ce726f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.411978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 380/743] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:29.412598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.444954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.445667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.445951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.446226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.456173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.457629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:29.475575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:29.476715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:29.477775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:29.477775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:29.477775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:29.477775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:29.477775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 364/744] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:29.477775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.533726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae21150-d562-4a56-857b-24d1b33e9ac9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.554405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.559563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.569742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.570409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.570694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.570975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.571796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:29.572111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.573536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.578520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d44bc1-c5a9-404d-8547-305552a1d6a7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.586248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 381/745] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.586956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.617542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f68d82-d7bd-4054-90b3-54aba6cd5a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.618195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f68d82-d7bd-4054-90b3-54aba6cd5a53 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.618481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f68d82-d7bd-4054-90b3-54aba6cd5a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.618754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f68d82-d7bd-4054-90b3-54aba6cd5a53 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.619504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f68d82-d7bd-4054-90b3-54aba6cd5a53 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.620866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f68d82-d7bd-4054-90b3-54aba6cd5a53 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.634444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 365/746] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:29 2022] POST /identity/v3/services => generated 240 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:29.635732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf52a3-6a81-49a5-9372-3e10f49c2126 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.636023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf52a3-6a81-49a5-9372-3e10f49c2126 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.636303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf52a3-6a81-49a5-9372-3e10f49c2126 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.637561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 366/747] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:29.638287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf52a3-6a81-49a5-9372-3e10f49c2126 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.712551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2a44f-e6ea-46eb-99bb-4377bd13a940 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.763119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.763863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.764160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.764446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.765287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:29.765530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.766393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf52a3-6a81-49a5-9372-3e10f49c2126 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.767556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.781885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 382/748] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.783400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.804057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.804723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.805011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.805990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.805990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:29.806305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.807719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62baf121-0c93-4f81-af14-97303829f50c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.810040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.822035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 367/749] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.823466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.823827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.824175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.833765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:29.842049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:29.843363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:29.901039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5ff71-a4ed-4485-b482-edcc3eafe2a5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:29.920476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 368/750] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:29.921953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:29.945933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:29.946610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:29.947235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:29.947541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:29.948467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=69edde6d461545069a9e98c661e5453e) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:29.948789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:29.950240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:29.957085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 383/751] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/role_assignments?user.id=69edde6d461545069a9e98c661e5453e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:29.957900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.042094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fe5bac-708d-4c07-9ba8-91b2e0c736e6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.072437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.083965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.084691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a462035-0b4f-4057-8b28-6664dda6752d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.084808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.085050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.085331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.087444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.097587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.099138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:30.118128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:30.119101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:30.120052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:30.120052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:30.120052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:30.120052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:30.120052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 369/752] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:30.120756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.128893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.130030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.130513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.131003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.139489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.141178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:30.157120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:30.158149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:30.159103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:30.161710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 384/753] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:30.162708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.252731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8630a5-e83d-4fa2-8bd7-17618387705b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.289589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.290398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.290674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.290944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.291736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.293183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:30.305788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:30.306775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:30.308050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 370/754] 10.222.0.43 () {56 vars in 1129 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:30.311952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.327613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54c32e7-ba4c-48e4-b0ec-c15ae6e2ad43 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.358238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.379308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.381026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.381518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.382007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.383256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:30.383774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.386220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.403643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 385/755] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:30.405220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.420291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf6f1e5-bc26-41a7-9f3a-231d5ef10425 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.438161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.466392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.467944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.468071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.468393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.469369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:30.469745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.471274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.490885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 371/756] 10.222.0.43 () {56 vars in 1149 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:30.491766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.529302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9bff85-7ffb-4884-b87a-ff213b46fc80 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.581572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.583805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.584454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.584977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.586895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:30.588065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.590655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.597021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.599771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 386/757] 10.222.0.43 () {56 vars in 1124 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:30.600200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7366f79-3d74-40f9-9535-808ad133317d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.601373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.638313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.639159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.639496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.639822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.648001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.649770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:30.665437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:30.666639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:30.667725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:30.671492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 372/758] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:30.672363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.718893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.739344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86852e75-600a-4a62-9512-1bbe7c782531 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.778589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.779550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.779993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.780390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.785864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f0c32-14d2-4753-b1c7-98c05f4d85d8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.823084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.824005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.824457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:30.824896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:30.825835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:30.826343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.827975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.834108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 373/759] 10.222.0.43 () {56 vars in 1124 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:30.835195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.857127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:30.858605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:30.868862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.955157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:30.961739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50359371-dd00-44f9-b173-ccc3dc7c50b5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:30.973204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 387/760] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:30.974166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:30.998240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:30.999267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:30.999758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.000269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.001856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:31.002373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.004021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.024690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 374/761] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:31.025935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.082383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efeb23e-2e76-44c6-b66f-c475413b95c4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.118958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:31.127124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.128618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.129138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.129873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.131039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] RBAC: Adding query filter params (interface=public, service_id=7326a09816b04f5e9a6083fc55e176ec, region_id=RegionOne) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:31.131982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.133972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.143260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 388/762] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:29:30 2022] GET /identity/v3/endpoints?service_id=7326a09816b04f5e9a6083fc55e176ec&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:31.144097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.144977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8d9756-f6d7-4c35-914c-a9afd5b5b2a5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.192607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.194420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.195110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.195844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.201054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:31.216092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.218096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:31.241927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:31.242901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:31.243872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:31.243872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:31.243872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:31.243872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:31.244889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 375/763] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:31.246430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.269262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b7bf7-d649-43c0-b41f-bcd9624f8796 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.305771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.306599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.306893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.307172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.322778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.325218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:31.349058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:31.350010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:31.350922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:31.350922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:31.350922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:31.350922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:31.353419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 389/764] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:31.354341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.369475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39443bcc-e4d0-482b-b26c-a53e983b7a9a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.406472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.407361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.407780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.408155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.408996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.410483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.411864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:31.417553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 376/765] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:31.418612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.484780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae228e0b-3923-4c2b-8ee7-2c55ec46cf18 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.488053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:31.538763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.541253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.541726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.542256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.544785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:31.547099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.549291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.578215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 390/766] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:31.584372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.592909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23250b3-a96b-4094-8ae9-66feec700fdd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.652048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.652968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.653404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.653829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.654717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:31.655183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.657281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.674323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 377/767] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:31.681328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.743165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4462d16d-a08c-4c08-b941-46f2df0d4d23 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.759635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:31.789446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.790256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.790679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.791106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.801268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.802827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.804327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5bc46d-af27-434c-aae4-86ad133a11ac None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-b3c7be24-072d-44b7-93aa-54d0a08a017d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:31.821937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:31.822888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:31.823852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 391/768] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:31.824829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c995ba9-17a4-468d-8b62-58adc08e748e None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.825103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c995ba9-17a4-468d-8b62-58adc08e748e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.825391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c995ba9-17a4-468d-8b62-58adc08e748e None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.827070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 392/769] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:31.832221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c995ba9-17a4-468d-8b62-58adc08e748e None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:31.854294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.855435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.855723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.856001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.861707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:31.865735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:31.867428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-31274740-a401-4f63-a558-cc2732be10a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:31.887569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:31.888537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:31.889447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:31.894220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 378/770] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:31.896278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.896567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.896848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.919854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:31.921251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:31.957130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c995ba9-17a4-468d-8b62-58adc08e748e None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:31.994683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:31.995823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:31.996313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:31.996842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:31.997857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:31.998489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.000194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.011286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 379/771] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:32.012822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.022396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 393/772] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:32.027117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.081413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.124643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.143106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e829226-316d-4d08-8e08-c6a02be5f6a4 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.147458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be79e12-bfcd-4572-9b20-1add02ba12b4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.181964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.182633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.182916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.183197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.184078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:32.184394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.185875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.189325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.189981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.190259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.190531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.191381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=5aa970dc0f084d6f9c46cb132964c02e) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:32.191693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.193549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec6b6d-524f-4390-b5c9-8eb7fe8419a6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.202416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 380/773] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:32.202416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.206529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 394/774] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/role_assignments?user.id=5aa970dc0f084d6f9c46cb132964c02e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:32.208545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495c57-6a6e-411b-b353-231441c97f44 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.208838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495c57-6a6e-411b-b353-231441c97f44 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.209118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5495c57-6a6e-411b-b353-231441c97f44 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.210703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 395/775] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:32.212013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5495c57-6a6e-411b-b353-231441c97f44 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.334034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5495c57-6a6e-411b-b353-231441c97f44 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.369905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55520d-05ac-45d7-ac49-049cbb2c7d59 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.373582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e55ca-e177-417d-930b-cb1598859367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.374746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e55ca-e177-417d-930b-cb1598859367 None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.375294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e55ca-e177-417d-930b-cb1598859367 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.375772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e55ca-e177-417d-930b-cb1598859367 None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/b502d4020b494807b09e803df0b2a301/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.378675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e55ca-e177-417d-930b-cb1598859367 None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=b502d4020b494807b09e803df0b2a301, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.380515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e55ca-e177-417d-930b-cb1598859367 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.383360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.400283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 396/776] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:32 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/b502d4020b494807b09e803df0b2a301/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:32.401849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.402270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.402680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.416129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.417086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.417430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.417797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] PATH_INFO: `/v3/services/network` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.418574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.420563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.421014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.430511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:32.432181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:32.436394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:32.437653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:32.438640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:32.441777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 381/777] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/services/network => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:32.447752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.563342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 397/778] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:32.595593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.613557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689dbb7-36ad-4e25-bf03-43e1ca2d865c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.664741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.665660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.666634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.667093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.668272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] RBAC: Adding query filter params (name=network) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:32.668971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.670512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.677152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 382/779] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:32.684728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.732116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4580e04-0e20-4d87-8197-281c23dcdbd3 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.738618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.777385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.778682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.779185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.779558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.782733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:32.799912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:32.807943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:32.808889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:32.809742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:32.812529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 398/780] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:32.820456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:32.873949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85c0b-eaf0-4e69-92f5-6bb996c1e654 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:32.928989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:32.930091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:32.930592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:32.931075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:32.932221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:32.932738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:32.934449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:32.940771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 383/781] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:32.948504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:33.010852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53f0e65-19df-4e9f-b9d7-891bbe1b3834 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:33.073370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:33.074438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.074932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.075466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.076442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:33.076957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:33.078543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:33.095873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 399/782] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:33.098394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:33.162860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:33.168802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18847afa-74df-4f79-9f4f-ce4b59a5f7c6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:33.181451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:33.232729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:33.233362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.233650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.233928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.234720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] RBAC: Adding query filter params (interface=public, service_id=1dd62470191442c1b403735310bd4016, region_id=RegionOne) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:33.235035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:33.238091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:33.251034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 384/783] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:29:32 2022] GET /identity/v3/endpoints?service_id=1dd62470191442c1b403735310bd4016&interface=public®ion_id=RegionOne => generated 194 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:33.258047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf5c34c-b284-4a99-aae3-441ca7ee567d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:33.260108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:33.301644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:33.302447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.302729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.303006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.315577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:33.317321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:33.347809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:33.348760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:33.349669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:33.349669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:33.349669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:33.349669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:33.351966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 400/784] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:33.367692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:33.415239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316674f-35c4-413a-9d7b-cfdafa691523 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:33.448543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:33.449215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.451138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.451457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.452160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:33.453562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:33.468228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 385/785] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:33 2022] GET /identity/v3/services => generated 772 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:33.549774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8ae1cc-50ee-433b-9c39-84d9d99dcd89 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:33.568923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:33.602772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:33.603489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.603771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.604044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.604787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:33.605093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:33.606489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eb1c7c-77bd-45eb-9cb9-5d890ba69c7e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:33.628260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 401/786] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:33.634105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:33.677952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:33.794093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8fc2e9-04d9-4cc3-b215-3bb54f9ae8cd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:33.842790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:33.843816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.844104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.844380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.852737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:33.852901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92716f9e-2003-413b-804c-04ffd30056b1 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.853117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92716f9e-2003-413b-804c-04ffd30056b1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.853381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92716f9e-2003-413b-804c-04ffd30056b1 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.854188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:33.855009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 402/787] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:33.859977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:33.860263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:33.860535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:33.869516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:33.870553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:33.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:33.875372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 386/788] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:33.877866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:33.900411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:33.903294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:34.021692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 403/789] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:34.028429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.042715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145282f-72fe-443e-b4ad-b0a76b1023f8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.044940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.117148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.117741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.118027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.118304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.119147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:34.119484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.120930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.146367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 387/790] 10.222.0.43 () {56 vars in 1135 bytes} [Fri Jun 24 08:29:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:34.155737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.186883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.191330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5036914e-c429-4a4d-aae5-99f999a1fe70 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.230177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.231087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.231614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.231961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.232996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.234489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:34.250035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:34.251306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 24 08:29:34.252870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:34.254203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 404/791] 10.222.0.43 () {56 vars in 1129 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:34.259857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.371708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b966356-0e1b-401b-8642-42ca9330bede None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.424536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.425337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.425619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.425896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.436589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.438049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.445807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:34.459313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:34.460238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:34.461149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:34.461149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:34.461149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:34.461149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:34.465187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 388/792] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:34.466363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.475914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd4e3b6-0189-4684-8811-bb7ff7479ed5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.556505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.557192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.557482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.557762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.558483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:34.558791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.560201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.573469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 405/793] 10.222.0.43 () {56 vars in 1124 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:34.574056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dc6c36-dca7-4cfa-b021-7fb7939fce0d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.578966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.611936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.612609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.612897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.613228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.613966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:34.614274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.615732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.632873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 389/794] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:34.633794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.675615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.697887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.717909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7572b9e-b64f-4dd0-b0e5-de79944f8211 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.744308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495e25a2-2ced-4070-8ff8-c614f1b7c774 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.753257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.753932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.754220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.754502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.755232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:34.755554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.756928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050f372-e0ed-40ce-b640-4cd2a7c4fd86 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.763258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 406/795] 10.222.0.43 () {56 vars in 1124 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:34.764561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b63b-78ab-44b0-b015-9c10c5b3dcfc None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.764852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b63b-78ab-44b0-b015-9c10c5b3dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.765138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6b63b-78ab-44b0-b015-9c10c5b3dcfc None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.766629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 407/796] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:34.769371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b63b-78ab-44b0-b015-9c10c5b3dcfc None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.781125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.781987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.782317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.783119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.793326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.793326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-6d30bfec-d9e5-4299-a678-ef3a14379249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:34.809383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:34.810324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:34.811240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:34.812825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 390/797] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:34.814190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.814474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.814748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.832341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:34.833567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:34.884476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.897813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6b63b-78ab-44b0-b015-9c10c5b3dcfc None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:34.906603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 391/798] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:34.908446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:34.962527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:34.963378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:34.963656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:34.964737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:34.965490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:34.966851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:34.978732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:34.995371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 408/799] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:29:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:34.997383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.040273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00dca0-f32d-487f-9911-3dadd46a0e05 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.086081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.086754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.087036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.087340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.088355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:35.088468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.089886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce7b74-9667-4c60-86d3-b387fde63bff None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.112270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 392/800] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:35.112270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa2c3c-c025-4356-bc76-582833da79a4 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.112270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa2c3c-c025-4356-bc76-582833da79a4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.112270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aa2c3c-c025-4356-bc76-582833da79a4 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.112270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 393/801] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:35.124828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa2c3c-c025-4356-bc76-582833da79a4 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.159986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b3a9e-360f-4093-9c40-f7341102a167 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.200873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f24380-d18d-4c2d-b77b-866212da3935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.201672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24380-d18d-4c2d-b77b-866212da3935 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.201955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24380-d18d-4c2d-b77b-866212da3935 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.202230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f24380-d18d-4c2d-b77b-866212da3935 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.215460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f24380-d18d-4c2d-b77b-866212da3935 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d3f24380-d18d-4c2d-b77b-866212da3935 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:35.223688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:35.224622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:35.225461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:35.228135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 409/802] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:35.229595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.230096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.230573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.247802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:35.249291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:35.252134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.270409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aa2c3c-c025-4356-bc76-582833da79a4 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.273399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.313373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.315092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.315421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.315703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/69edde6d461545069a9e98c661e5453e/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.318333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=69edde6d461545069a9e98c661e5453e, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.320240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.330312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 410/803] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:35.331718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.351323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 394/804] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:35 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/69edde6d461545069a9e98c661e5453e/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:29:35.365152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.529067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37786a54-7b24-42bb-8889-0252d5996d00 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.556895 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b6a80-fb5e-466e-b9a4-9fe2571e957c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.569569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.570240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.570520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.570796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.571595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:35.571901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.573266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.580503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.587088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 411/805] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:35.592577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.603520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.605208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.605716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.606614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.622133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:35.633600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:35.634572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:35.635454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:35.641383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 395/806] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:35.649379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.691893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.770537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c41562-be23-4a7c-aab2-cc48f5354e90 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.785227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747a5ae-0c9c-44d0-898e-33027c24231b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.806567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.807277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.807566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.807839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.808596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:35.808900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.810298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.823328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 396/807] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:35.825419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.833728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.834558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.834818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.835099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.845728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.847199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:35.865548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:35.866498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:35.867474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:35.867474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:35.867474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:35.867474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:35.867474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 412/808] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:35.872711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:35.917594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.935042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1128ce-94f1-46f8-8cb2-f3dae3e63421 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:35.937489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:35.972615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:35.973696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:35.974140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:35.974447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:35.985075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:35.987055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-c492c4a2-b180-4394-8be0-372eedbc9482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:36.005273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:36.006386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:36.007483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:36.007483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:36.007483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:36.007483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:36.011087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 397/809] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:36.012425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92408d-df55-4fa6-b4b4-93688ac3638e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.012715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92408d-df55-4fa6-b4b4-93688ac3638e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.012998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc92408d-df55-4fa6-b4b4-93688ac3638e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.013415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126537ae-d4d0-4098-b11c-e77a3124bfb7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.015043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 398/810] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:36.016003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92408d-df55-4fa6-b4b4-93688ac3638e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.059049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.059748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.060029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.060305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.061537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:36.061842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.064188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c00c777-8176-4004-ae77-425e923afc48 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.082894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 413/811] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:36.084338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.084637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.084926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.101986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:36.103179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:36.134641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc92408d-df55-4fa6-b4b4-93688ac3638e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.175296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.176354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.176816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.177264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.178230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:36.178688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.180355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.182149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 414/812] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:36.183080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.190300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.200025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 399/813] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:36.201490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.248098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.309642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420db0f5-5aae-48ef-bec4-15f3150303f1 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.316094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb448ee-c867-4bfa-a7dc-36ce70925994 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.346996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.347965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.348253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.348540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.356755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.357049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.358019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.358449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.358589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.358994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.367796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.369695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:36.374965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:36.376182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:36.378026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:36.381424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 415/814] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:36.382166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:36.389637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:36.390780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:29:36.392072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:36.398683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 400/815] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:36.399940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.467079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.490721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd88bd-bf56-4699-b7ff-590ec54702d8 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.492307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.516145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c3a0b6-a5e1-418e-9f86-121d0663125b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.528472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.529277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.529576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.529843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.530501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.532011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:36.544469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:36.545632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:36.546612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 416/816] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/services/network => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:36.547511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.556997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.558239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.558694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.559118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.560203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:36.560664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.562371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.584006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 401/817] 10.222.0.43 () {56 vars in 1149 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:36.585219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.665543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab5250-e6c6-44ef-abfa-084bb52f90a9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.700139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5985e39-2d0f-4ad9-b284-5ffb44387ae7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.702709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.704246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.704246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.704246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.704954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:36.705265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.706716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.721358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.728138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 417/818] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:36.728138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.746728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.747692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.748211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.748632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.749535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:36.749990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.751678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.755705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.757968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 402/819] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:36.759270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.837640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368e25ae-5b41-4241-b212-130ddae89136 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.870027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d2c1d-3056-4eee-a9f3-98510eae8cc2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:36.873678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.874488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.874765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.875052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.885881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.887572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:36.905391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:36.906397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:36.907908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:36.907908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:36.907908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:36.907908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:36.911785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 418/820] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:36.912397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:36.912607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:36.913418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:36.913880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:36.914332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:36.971616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.993868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:36.994354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:36.996766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.017290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d5fffb-34dd-4fde-9710-8f49ef00c7fe None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.053866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.054222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.054514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.054793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.055540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] RBAC: Adding query filter params (type=network) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:37.055853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.057227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.066925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 419/821] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/services?type=network => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:37.067865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.112926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 403/822] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:37.114281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.173144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abe079b-cbbc-43e2-b360-2aadb224102f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.209389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.210141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.210343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.210625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.211401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:37.211713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.213085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.219586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.222347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.230688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 420/823] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:37.231605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.231918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548eb1cd-6b4f-4613-b29c-8a5caaf7cd30 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.271264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.272636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.273295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.273789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.275002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.276779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.300727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 404/824] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:29:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:37.301866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.364251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb7f5-6f7c-4623-855b-5802ec8741db None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.416112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.417297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.417888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.418393 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.432348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.432548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12d4565-f7c7-44b8-a26a-df7e5380234a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.434397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-bf574a1d-d631-4ef4-a73c-a2805d6f0d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:37.454064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:37.455652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:37.456567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:37.456567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:37.456567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:37.456567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:37.458746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 421/825] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:37.460396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659c3fc-1948-4676-827c-8e15acdadc33 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.460745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659c3fc-1948-4676-827c-8e15acdadc33 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.461088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f659c3fc-1948-4676-827c-8e15acdadc33 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.462537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 422/826] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:37.464186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659c3fc-1948-4676-827c-8e15acdadc33 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.481903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.482715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.483190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.483364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.486578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.493480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.494946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-2ec0bbff-82b0-46e4-a51c-d2874a5fc8e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:37.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:37.516718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:37.518494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:37.520583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 405/827] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:37.522246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.522693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.523115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.543435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:37.544929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:37.643887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.698230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f659c3fc-1948-4676-827c-8e15acdadc33 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.731984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 406/828] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:37.732489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.732815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.733018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.733354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.733634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.734383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:37.734690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.736116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffd968-b2b2-47e5-9138-552728db5a7a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.747255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 423/829] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:37.748515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9a888-2a1b-4bcd-9f84-c18a41c38bfd None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.748805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9a888-2a1b-4bcd-9f84-c18a41c38bfd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.749083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9a888-2a1b-4bcd-9f84-c18a41c38bfd None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.750580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 424/830] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:37.753221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9a888-2a1b-4bcd-9f84-c18a41c38bfd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.845624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329432f-dc46-468b-a89c-824bc7363a14 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.852346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9a888-2a1b-4bcd-9f84-c18a41c38bfd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:37.861642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.872631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:37.882537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.883200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.883783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.884063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.884850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:37.885163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.886606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d816a-ce11-48e8-8bbb-06b54b3f66d2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.887488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:37.888269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.888546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.888814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.896001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:37.897352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:37.898056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 407/831] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:37.899520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:37.899805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:37.900078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:37.915312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:37.915648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:37.916762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:37.918005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:37.918444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:37.918524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 425/832] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:37.918524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:37.979786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 408/833] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:37.980577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.020508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484b0f-b8d7-4afc-80ba-b779c2f4bc04 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.053527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.054324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.054611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.054891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.055575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.056930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:38.069140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:38.070182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:38.071047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 426/834] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:38.071911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.084444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28cf200-184f-493e-8a99-0d13486f6361 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.100907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.105308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.119381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.120265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.120544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.120815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/5aa970dc0f084d6f9c46cb132964c02e/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.135383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=5aa970dc0f084d6f9c46cb132964c02e, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.136815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.148958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 409/835] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:37 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/5aa970dc0f084d6f9c46cb132964c02e/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:38.149698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.192103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8b484-f078-4224-937d-a19705924db9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.232641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.233314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.233597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.233868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.234769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:38.235080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.237126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.255759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 427/836] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.255759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.269414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91df518-c503-4a8d-8eb4-3b9b290226b7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.312439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.313279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.313562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.314534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.325603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:38.333802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:38.334943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:38.335981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 410/837] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:38.336840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.362218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.363588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.375619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305393c-2d5b-460c-b670-9db685eaf8f7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.417314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.417907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.418190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.418470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.419183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:38.419528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.420900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.425778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 428/838] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.427160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.444946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a086ed-0a90-476b-8df4-03d9554253a2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.478915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.479876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.480306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.480738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.481578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=b502d4020b494807b09e803df0b2a301) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:38.481894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.483352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.489250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 411/839] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/role_assignments?user.id=b502d4020b494807b09e803df0b2a301&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.490036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.559012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10061431-71e6-4343-88bf-7b9100e2a4d4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.606854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.615544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.617180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.617745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.618303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.619283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:38.619768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.619885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31461fb6-1ff7-4a41-a812-becd1c924d3f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.621803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.635384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 429/840] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.640896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.658366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.659260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.659620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.659902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.660688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:38.661062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.662453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.668461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 412/841] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:38.672728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:38.771117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be4f9e-9b12-4e8d-a777-3fc74c250946 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.811839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.812652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.812932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:38.813207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:38.825886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:38.827368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:38.845327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:38.846343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:38.847700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:38.847700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:38.847700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:38.847700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:38.847700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 430/842] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:38.852229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:38.963617 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25669ce1-3864-427b-b53c-7459327f0abc None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:38.998771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:38.999466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:38.999745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:39.000017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:39.000740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:39.001043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:39.002933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:39.015552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 413/843] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:39.021610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:39.103785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:39.138554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84770d-5370-4855-9b75-d5fda2e06f36 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:39.175967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:39.176772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:39.177053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:39.177327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:39.185689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:39.187114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:39.201578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:39.202524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:39.203511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 431/844] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:29:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:39.209828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:39.226066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:39.315036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca63cdd2-0fdb-41f1-80f0-c2bc1d27205c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:39.350118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:39.350788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:39.351066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:39.351367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:39.352208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:39.352512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:39.353945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:39.372997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 414/845] 10.222.0.43 () {56 vars in 1135 bytes} [Fri Jun 24 08:29:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:39.380006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:39.496675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e20cb0-63d5-4d51-a451-b8c6bf8dcf42 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:39.525943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:39.531963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:39.532853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:39.533321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:39.533761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:39.604528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:39.606323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:39.709859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 432/846] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:39.715257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:39.790713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:39.825518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df527-b11e-4784-8c76-3622e309f900 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:39.870321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:39.871130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:39.871481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:39.871763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:39.883111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:39.884703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:39.902864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:39.903823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:39.904745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:39.904745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:39.904745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:39.904745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:39.904745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 415/847] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:39.910426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:39.941444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.011351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd19dca-cbd7-4185-bf01-e56b148354e5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.045683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.046278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.046556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.046830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.047578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:40.047876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.049828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.061527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 433/848] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:40.066949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.185551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75928c6-0a01-4c41-aff0-caaf3e0e83e6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.210424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.222141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.223113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.223657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.224099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.231347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.232807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-585607b6-2996-4106-83e4-d1dc4da83bd6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:40.247269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:40.248167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:40.249087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 416/849] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:40.254228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.270368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f266-919d-4a8d-b1bd-30150996c7b5 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.270656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f266-919d-4a8d-b1bd-30150996c7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.270929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9723f266-919d-4a8d-b1bd-30150996c7b5 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.272676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 417/850] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:40.277792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.278075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.278353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.298405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:40.299558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:40.361438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.373611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd22e619-8d9d-42d6-89d9-48ec16498e07 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.387290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 418/851] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:40.412157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.418263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.419662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.420212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.420746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.421792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:40.422355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.424120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.438303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 434/852] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:40.445066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.513787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5d2b5-642d-4a06-af34-e005dfb532bb None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.543666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.551886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.552612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.552890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.553171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.563050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9299dd-ad4e-4f2e-af9d-4e38d26f9581 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.565735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:40.573693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:40.574660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:29:40.575563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:40.579490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 419/853] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:40.580848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.605134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.606167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.606797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.607297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.608430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=69edde6d461545069a9e98c661e5453e) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:40.608959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.611458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec16269-cc07-40c1-8d2b-6b665e11a928 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.623362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 435/854] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/role_assignments?user.id=69edde6d461545069a9e98c661e5453e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:40.685122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae4e9f-becb-4026-95dc-9b5d3f29b009 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.685784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae4e9f-becb-4026-95dc-9b5d3f29b009 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.686290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ae4e9f-becb-4026-95dc-9b5d3f29b009 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.688346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 436/855] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:40.692530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1512259-c707-4456-85eb-af7f3edce8d7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.694477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.694776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.695058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.713011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:40.714421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:40.727906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.731018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.731727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.732010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.732283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.733041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:40.733346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.734729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.744720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 420/856] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:40.750678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.795254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 437/857] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:40.801197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:40.817859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.873029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f31d3e-9bae-4192-98ec-2165ff53cdc9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.912648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.914110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.914771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.915237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.924885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b724-3c9d-4a5b-b224-232c44580bd2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:40.929422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.931013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-832ae5f2-15cb-43d2-96f6-d74657828204 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:40.952953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:40.954089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:40.955754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:40.955754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:40.955754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:40.955754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:40.961421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 421/858] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:40.961421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ebc93-0c3d-4951-96d3-3721e73ec0d8 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.961421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ebc93-0c3d-4951-96d3-3721e73ec0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.961421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ebc93-0c3d-4951-96d3-3721e73ec0d8 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.961421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 422/859] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:40.964192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ebc93-0c3d-4951-96d3-3721e73ec0d8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:40.966154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:40.967043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.967434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.968375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:40.968555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:40.970577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-50a30d85-be8b-4845-829c-de5e19fb973d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:40.983074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:40.984083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:40.984953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 438/860] 10.222.0.43 () {56 vars in 1132 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:40.987169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:40.987480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:40.987761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.004812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:41.006304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:41.008732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.067507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.087305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 439/861] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:41.087305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.091792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ebc93-0c3d-4951-96d3-3721e73ec0d8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.132249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.132913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.133197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.133473 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.134204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:41.134511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.135990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.149040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 423/862] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:41.150213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.194694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2807f049-1f00-4397-9506-a36786b5ee07 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.230785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.231974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.231974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.231974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.232687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:41.232984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.234383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.239541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 440/863] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:41.240323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.242921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.268370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa330b66-177a-41ae-87d9-00d4630ba1e9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.304220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.305040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.305424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.305803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.306629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.308126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.323448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 424/864] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:41 2022] POST /identity/v3/services => generated 251 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:41.324389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.337668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.363875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f783fc0-980f-4ba5-942c-3134ed7cddf9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.413546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.414588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.415020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.415422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.424385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.426046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:41.443096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:41.444356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:41.445537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:41.452074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 441/865] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:41.453507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.457273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2728fd3-16f7-4535-b288-c6ab5eb172af None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.495966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.496643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.496921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.497191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.498345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:41.498658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.501626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.506352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 425/866] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:41.524464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.576847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9816fe-fb4d-424a-945f-da184fedf96f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.616909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.617508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.617791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.618063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.618908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:41.619226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.620677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.641346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 442/867] 10.222.0.43 () {56 vars in 1149 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:41.647114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:41.727833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.761593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb4640-7070-4ff2-b096-54d3dd1829fe None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:41.798719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:41.799774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:41.800238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:41.800702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:41.872977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:41.874823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:41.948040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:41.980687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 426/868] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:41.987325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:42.092307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097f358-c480-4277-b75d-76a32b546abc None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:42.110213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:42.128772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:42.129506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.129989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.130268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.140861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:42.142335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:42.160380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:42.161307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:42.162624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:42.162624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:42.162624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:42.162624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:42.162624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 443/869] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:42.172334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:42.278223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c18029b-f100-4589-b8fb-f1b0b06d0ef7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:42.313422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:42.314121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.314401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.314676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.315432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:42.315736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:42.317105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:42.329082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 427/870] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:42.334928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:42.436241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c0d0b8-196e-4542-a74a-5cdc8fdb0b84 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:42.470932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:42.471754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.472036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.472312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.479237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:42.480680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:42.494495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:42.496672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:42.498866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 444/871] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:42.502192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:42.536502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:42.609918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d25d1-6fd1-4c59-80fb-f1dd081487d2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:42.644989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:42.645654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.645937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.646215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.647003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:42.647340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:42.648782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741482d-a5a1-4e9a-bbc5-c4c6de38cac5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:42.661465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 428/872] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:42.667276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:42.697903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:42.715337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52bcb8-9393-464f-a8ba-9243fc9b8e72 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.715761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52bcb8-9393-464f-a8ba-9243fc9b8e72 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.716045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52bcb8-9393-464f-a8ba-9243fc9b8e72 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.718428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 429/873] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:42.724777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.725060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.725337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.742105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:42.743314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:42.782406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf8f44a-1a48-4f57-96d3-36b6b4c3446f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:42.817401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:42.818098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:42.818379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:42.818701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:42.819556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=5aa970dc0f084d6f9c46cb132964c02e) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:42.819862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:42.821357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:42.823296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 430/874] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:42 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:42.832709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 445/875] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:42 2022] GET /identity/v3/role_assignments?user.id=5aa970dc0f084d6f9c46cb132964c02e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:42.845487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:42.879664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:42.959991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165eb65-8fde-4ee7-b434-93304258cac8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:42.999163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:43.000307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.000773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.001783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.002635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:43.004271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:43.017475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:43.018509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:43.019407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:43.021860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 431/876] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:42 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:43.028638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:43.128070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:43.138183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf8a8ba-1db9-413b-833d-108901324d4d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:43.177666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:43.178354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.178640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.178924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.179667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:43.179977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:43.183422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4572f63b-6462-49e7-af94-87ad509c8707 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:43.187863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 446/877] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:43.195231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:43.282687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:43.297768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175da2d-b08d-4a74-b872-4254b6b2caf0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:43.333911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:43.334587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.334871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.335148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.335890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:43.336199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:43.337555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b57615b-f1aa-4b44-b106-375d6ac8aa0c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:43.342509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 432/878] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:43 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:43.504185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aae319-7db6-4f42-9989-f13d51a8cf8b None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.504691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aae319-7db6-4f42-9989-f13d51a8cf8b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.504770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aae319-7db6-4f42-9989-f13d51a8cf8b None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.506272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 447/879] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:43.511630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0f0f-61ac-49c9-a7c5-ab00b9f287b9 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.511921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0f0f-61ac-49c9-a7c5-ab00b9f287b9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.512199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980e0f0f-61ac-49c9-a7c5-ab00b9f287b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.529314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-980e0f0f-61ac-49c9-a7c5-ab00b9f287b9 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:43.530491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-980e0f0f-61ac-49c9-a7c5-ab00b9f287b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:43.598792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 433/880] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:43.621839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aae319-7db6-4f42-9989-f13d51a8cf8b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:43.722571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13d681-f3eb-401d-b1fc-aa7827190148 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.722899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13d681-f3eb-401d-b1fc-aa7827190148 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.723123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a13d681-f3eb-401d-b1fc-aa7827190148 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.724789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 434/881] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:43.729544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.729820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.730100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.736180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51aae319-7db6-4f42-9989-f13d51a8cf8b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:43.747792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:43.748838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:43.774080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:43.775490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.776190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.777073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.778164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:43.780015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:43.795007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:43.796262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:43.797304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:43.801339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 448/882] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:43.807142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:43.835578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:43.840091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 435/883] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:43 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:43.862862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:43.878488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:43.922178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0aa8c9-ab09-4396-a960-e020a81ba706 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:43.960698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:43.961707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:43.962191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:43.962664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:43.964044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:43.964558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:43.966510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:43.977969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 449/884] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:43.979872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:43.996465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2959413-e3d4-4e0f-9b3d-0d82ebbbf769 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:44.033441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:44.034493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:44.034985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:44.035494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:44.036438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:44.037992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:44.063576 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 436/885] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:43 2022] POST /identity/v3/services => generated 269 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:29:44.084350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:44.104702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb4a55-2a14-45c4-8eb8-54e9cad3b40b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:44.151008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:44.151690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:44.151978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:44.152306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:44.153031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:44.153339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:44.154787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:44.161439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 450/886] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:44.168487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:44.249737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:44.284244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1522b1ea-0488-4e6b-8a9c-0325d34404ed None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:44.320108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:44.320764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:44.321044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:44.321312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:44.322096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] RBAC: Adding query filter params (interface=public, service_id=12962fe3e014401dbac324e8cfc163f9, region_id=RegionOne) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:44.322398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:44.323905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4669c-7f15-41f8-be44-18d35bf370ec None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:44.330028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 437/887] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:29:44 2022] GET /identity/v3/endpoints?service_id=12962fe3e014401dbac324e8cfc163f9&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:44.335500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:44.440802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed69bfea-e8fd-4f4e-8a0a-362f43ec60ed None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:44.476197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9453b6-f1e1-41c8-b095-885f4d7115e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:44.476868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9453b6-f1e1-41c8-b095-885f4d7115e9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:44.477190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9453b6-f1e1-41c8-b095-885f4d7115e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:44.477463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9453b6-f1e1-41c8-b095-885f4d7115e9 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:44.478194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9453b6-f1e1-41c8-b095-885f4d7115e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:44.479643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9453b6-f1e1-41c8-b095-885f4d7115e9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:44.485946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 451/888] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:44 2022] GET /identity/v3/services => generated 1268 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:29:45.001558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c6c2-bea0-4a7e-a7c8-03f2118bfee5 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.002078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c6c2-bea0-4a7e-a7c8-03f2118bfee5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.002192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2c6c2-bea0-4a7e-a7c8-03f2118bfee5 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.003732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 438/889] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:45.008833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.009647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.010125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.028011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:45.029500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:45.031892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:45.101267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 452/890] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:45.124138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2c6c2-bea0-4a7e-a7c8-03f2118bfee5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:45.238595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2c6c2-bea0-4a7e-a7c8-03f2118bfee5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:45.274087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:45.275504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.276005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.276475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.278602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:45.289191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:45.297582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:45.298675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:45.299980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 439/891] 10.222.0.43 () {56 vars in 1120 bytes} [Fri Jun 24 08:29:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:45.305353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:45.413400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eef9a1-7ee5-4093-9cfc-de75095b2d9d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:45.448920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:45.449589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.449869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.450141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.450882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:45.451184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:45.452559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8113260-cf0a-4b4f-bd66-359af71daa0c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:45.457748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:45.462160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 453/892] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:29:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:45.467615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:45.488765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e507bb-408f-4e59-b8e1-91a1009a4d7c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.489061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e507bb-408f-4e59-b8e1-91a1009a4d7c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.489318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e507bb-408f-4e59-b8e1-91a1009a4d7c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.490822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 454/893] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:45.496399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.496705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.496962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.514471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:45.515674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:45.594167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 455/894] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:45.605243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8adf2-5b57-4563-b9f0-e74d69620503 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:45.612955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:45.650856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:45.651830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.652212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.652591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.664166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:45.665738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:45.685449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:45.686593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:45.687720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:45.687720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:45.687720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:45.687720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:45.694039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 440/895] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:45.695037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:45.742455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5edb070-e8b9-4fe6-b5ed-6ff97af6044d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:45.757637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:45.775648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:45.785294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:45.787077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.787606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.788085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.789028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:45.791097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:45.808514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 456/896] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:45 2022] POST /identity/v3/services => generated 241 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:29:45.824362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9d7a1-ba52-4b0d-910e-b17429803264 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:45.873890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:45.874931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:45.875447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:45.875973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:45.877049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:45.877622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:45.879404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:45.893440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 441/897] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:45.899831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.011126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406dc39-c3fd-4045-b174-e9f3dc6e7abf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.047594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.049349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.049851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.050332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.058797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.060522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:46.075599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:46.076585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:46.077542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:46.080747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 457/898] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:46.086561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.194409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e294033-4387-4e71-83a6-14c09887df26 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.195945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:46.230948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.231647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.231930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.232204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.233032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:46.233336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.234759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c93e4-6d54-445d-b21a-2d5b851c7ca2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.252797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 442/899] 10.222.0.43 () {56 vars in 1149 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:46.258518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.309521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ab48-82e4-451e-841e-2db2c6a33331 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.309856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ab48-82e4-451e-841e-2db2c6a33331 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.310090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009ab48-82e4-451e-841e-2db2c6a33331 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.311776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 443/900] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:46.313394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:46.317540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.318128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.318128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.337266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:46.340703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:46.371905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b610da-158b-4b43-9435-8b08d775bbbd None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.408462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65e7572-41fb-4b5d-9a76-97dd10cb559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.409411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e7572-41fb-4b5d-9a76-97dd10cb559b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.409854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e7572-41fb-4b5d-9a76-97dd10cb559b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.410131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65e7572-41fb-4b5d-9a76-97dd10cb559b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.431451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 444/901] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:46.455673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.490530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e7572-41fb-4b5d-9a76-97dd10cb559b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.492030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65e7572-41fb-4b5d-9a76-97dd10cb559b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.550008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:46.586172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:46.596916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7583b178-e5a5-4987-b2d5-0f6d789d1a7b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.611112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 458/902] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:46.611112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c1f7a-c879-414e-8fce-fc93d433ec23 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.611112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c1f7a-c879-414e-8fce-fc93d433ec23 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.611112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c1f7a-c879-414e-8fce-fc93d433ec23 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.615301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 459/903] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:46.615301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c1f7a-c879-414e-8fce-fc93d433ec23 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.632350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.633160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.633440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.633714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.634374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.635782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-c15bb2e6-423c-4214-b120-98f364502fa1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:46.648736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:46.650186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:46.651478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 445/904] 10.222.0.43 () {56 vars in 1132 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:46.652376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.652376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.652376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.668486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:46.669665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:46.731334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c1f7a-c879-414e-8fce-fc93d433ec23 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.738560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 446/905] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:46.740207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.767796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.768847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.768847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.769122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.780100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.781586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:46.799915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:46.800843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 460/906] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.806951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:46.842480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:46.860102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c2e755-e203-4848-a814-043ee5b27619 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.897368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.898036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.898319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.898595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.899984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:46.900298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.901664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.906605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 447/907] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:46.908135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:46.912364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232bcfd7-cb4f-4af2-a740-12a9cdaf29e1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:46.948130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:46.948924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:46.949206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:46.949481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:46.950132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:46.951514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:46.964097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:46.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:46.966025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 461/908] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:46.967918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.020991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8352b-1723-47f7-8c4f-ebf9f9704144 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.039448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.054978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.057763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.058046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.058322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.059052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.059379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.060765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.072432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 448/909] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.073238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.083763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.085004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3cbfd-ed76-44e6-8230-5f34a067cdcf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.125470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.126677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.126961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.127258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.128378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.128690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.130048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.139364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 462/910] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:46 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.140464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.180375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5bf8d-c64c-4c9f-a475-5a223f8e1dec None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.215376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.215652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.215964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.216238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.216936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.217247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.218689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.227671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 449/911] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.229281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.251269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477a21-19c9-4766-8dfb-cfa5b3bc1010 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.281768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.287740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.288640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.288958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.289227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.296466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.298681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.301929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:47.313779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:47.314895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 463/912] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:47.315892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.337299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540b945-5a9a-4953-bd32-34d0f0f6476d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.372757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.373409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.373691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.373964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.374754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] RBAC: Adding query filter params (interface=public, service_id=2725714b4aa741a2a93b80246901f003, region_id=RegionOne) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.375058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.376503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.386030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 450/913] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/endpoints?service_id=2725714b4aa741a2a93b80246901f003&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.386030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.429749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4e7fc-5b9a-43dd-babc-39889636f781 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.465533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.466187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.466469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.466745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.467466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.467861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.469346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.479599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 464/914] 10.222.0.43 () {56 vars in 1114 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.480472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.483920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c10fa-b4cd-4513-b666-b28993c76969 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.517928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.518869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.519390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.519840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.520804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.521184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.522625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.525502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.537071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 451/915] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.537947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.538065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.593788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14e36b5-3a99-4ee5-9691-8734211b2d7d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.629881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.630472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.630756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.631034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.631822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.634233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.644700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 465/916] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/services => generated 1497 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:29:47.645314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:47.661824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6f4af-b7b8-49eb-9c79-cb9182afc661 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.715477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cece91-1163-4bd5-90c6-1e136355797c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.716156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cece91-1163-4bd5-90c6-1e136355797c None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.716440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cece91-1163-4bd5-90c6-1e136355797c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.716715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cece91-1163-4bd5-90c6-1e136355797c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.717474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cece91-1163-4bd5-90c6-1e136355797c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.718976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cece91-1163-4bd5-90c6-1e136355797c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.745990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 452/917] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:29:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:47.781701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4babfd-d8d6-4f3f-864a-8fb92be8f32d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:47.805490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:47.833715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:47.834804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.835324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.835841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.836938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] RBAC: Adding query filter params (role.id=86faf44e636e4eb283d98058960b8ee1, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=5aa970dc0f084d6f9c46cb132964c02e) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:47.837484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:47.839158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c256ee-d8ac-4b44-be59-0fea45f7adfd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:47.847483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 466/918] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:47 2022] GET /identity/v3/role_assignments?user.id=5aa970dc0f084d6f9c46cb132964c02e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=86faf44e636e4eb283d98058960b8ee1 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:47.957671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564e09c-e1d1-4c52-bcfb-2d1954e034f7 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.958014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564e09c-e1d1-4c52-bcfb-2d1954e034f7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.958228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9564e09c-e1d1-4c52-bcfb-2d1954e034f7 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.959937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 453/919] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:47.965588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:47.965898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:47.966155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:47.984671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:47.985935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:48.058672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 467/920] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:48.082847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564e09c-e1d1-4c52-bcfb-2d1954e034f7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:48.138826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:48.200815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9564e09c-e1d1-4c52-bcfb-2d1954e034f7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:48.236472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:48.237301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:48.237581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:48.237851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:48.238512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:48.239994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:48.253109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:48.254151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:48.255010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 454/921] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:48.260391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:48.364103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc53a2d5-1c2d-4667-9a1e-6024d08980be None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:48.373760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:48.400769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:48.401439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:48.401719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:48.401997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:48.402699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:48.403002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:48.404388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:48.409587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 468/922] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:48.415248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:48.522761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ce1fe-11c2-4517-be33-291c88a77e6d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:48.539566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:48.558872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:48.559578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:48.559859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:48.560129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:48.560838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:48.561141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:48.562572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:48.568492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 455/923] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:48.574500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:48.684095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf384e0-841f-48e3-8f5c-eb7525dc4c3b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:48.721147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:48.721818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:48.722106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:48.722387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:48.723183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] RBAC: Adding query filter params (interface=public, service_id=3e6a4abab23b4da89f4d5123e33ede46, region_id=RegionOne) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:48.723523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:48.725005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3170e43d-5724-48d1-8cb6-4cfe75be4ee4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:48.730713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 469/924] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:29:48 2022] GET /identity/v3/endpoints?service_id=3e6a4abab23b4da89f4d5123e33ede46&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:48.736494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:48.842703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258a3821-4ffd-4f26-9f3c-433cc2039a78 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:48.878914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8a8b2-1943-474b-b735-28bd64afc481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:48.879944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a8b2-1943-474b-b735-28bd64afc481 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:48.880436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a8b2-1943-474b-b735-28bd64afc481 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:48.880943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8a8b2-1943-474b-b735-28bd64afc481 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:48.881828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8a8b2-1943-474b-b735-28bd64afc481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:48.883415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8a8b2-1943-474b-b735-28bd64afc481 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:48.889992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 456/925] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:48 2022] GET /identity/v3/services => generated 1497 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:29:49.814797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40490eab-f485-4c0c-af98-89f49138724c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:49.815345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40490eab-f485-4c0c-af98-89f49138724c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:49.815426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40490eab-f485-4c0c-af98-89f49138724c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:49.817028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 470/926] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:49.822193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792be419-7ac5-4648-a008-7c63077e481b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:49.822641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792be419-7ac5-4648-a008-7c63077e481b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:49.822820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792be419-7ac5-4648-a008-7c63077e481b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:49.840073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-792be419-7ac5-4648-a008-7c63077e481b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:49.841259 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-792be419-7ac5-4648-a008-7c63077e481b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:49.881509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:49.909818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 457/927] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:49.932548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40490eab-f485-4c0c-af98-89f49138724c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:49.950776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ec25-2685-4b17-af8b-f9bda68af8d3 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:49.951077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ec25-2685-4b17-af8b-f9bda68af8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:49.951383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ec25-2685-4b17-af8b-f9bda68af8d3 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:49.953001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 458/928] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:49.958352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec235b8-7ccb-48eb-af31-6e04e9bdf597 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:49.958639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec235b8-7ccb-48eb-af31-6e04e9bdf597 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:49.958920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec235b8-7ccb-48eb-af31-6e04e9bdf597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:49.974958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:49.979379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-dec235b8-7ccb-48eb-af31-6e04e9bdf597 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:49.981048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dec235b8-7ccb-48eb-af31-6e04e9bdf597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:50.050205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40490eab-f485-4c0c-af98-89f49138724c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.053535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 459/929] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:50.056594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4b30b-48ea-48c3-a78a-f7534ef1591b None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.057102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4b30b-48ea-48c3-a78a-f7534ef1591b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.057604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4b30b-48ea-48c3-a78a-f7534ef1591b None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.059944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 460/930] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:50.065754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.065975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.066250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.085809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:50.086990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:50.095624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.096680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.097168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.097630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.098490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.100060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:50.112692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:50.113860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 24 08:29:50.114922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.116279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 471/931] 10.222.0.43 () {56 vars in 1132 bytes} [Fri Jun 24 08:29:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:50.117078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.167546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 461/932] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:50.168447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.206160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.248385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.252395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6e4fa5-e1b6-4841-a908-51dd7c218fcd None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.279568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aacc28-fc60-4682-88b5-24cc50325c77 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.288011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.288823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.289110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.289392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.290222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.291696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.293310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING keystone.server.flask.common [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 08:29:50.311743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.sql.core [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 08:29:50.311743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 08:29:50.311743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [parameters: {'id': '25ce8323024e4a2da392541c1cb7ac4e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 08:29:50.311743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80406) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 08:29:50.314168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.314835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.315137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.315449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.316202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:50.316460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.317919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 25ce8323024e4a2da392541c1cb7ac4e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 25ce8323024e4a2da392541c1cb7ac4e. Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:29:50.320355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result.read() Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 24 08:29:50.321181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application yield resource Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise self.value Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application session.commit() Jun 24 08:29:50.322185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.session.flush() Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application compat.raise_( Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 08:29:50.323701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 08:29:50.324632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application result.read() Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:29:50.325430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [parameters: {'id': '25ce8323024e4a2da392541c1cb7ac4e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.326384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:50.327161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 25ce8323024e4a2da392541c1cb7ac4e. Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 472/933] 10.222.0.43 () {60 vars in 1140 bytes} [Fri Jun 24 08:29:50 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:50.328288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.336470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 462/934] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:50.336470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.434580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1962c1-4775-4dc3-a29b-b2c45f03458a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.440122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.446391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714576c4-b545-4c05-8617-47d5d2f63748 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.471468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.472894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.473183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.473461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.482072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.482861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.483174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.483448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.485602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:50.490237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.491718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:50.494695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:50.495914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 473/935] 10.222.0.43 () {56 vars in 1120 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:50.502433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:50.505785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:50.506696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:50.507662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 463/936] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:50.508179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.524123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.615569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422ff66-b3c9-49d0-a650-b143af6d5ae3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.621638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e04e76b-871a-4d0c-8e01-4d2a5dc306d3 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.651900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.652867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.653339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.653689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.654398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:50.654706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.656293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.657340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.657999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.658284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.658561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.659335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:50.659642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.660995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.665142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 474/937] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:50.666118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.673637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 464/938] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:50.674648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.698558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.764144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.784420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ad7e7e-1312-4c2d-87d8-38a4e05e1fdc None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.787799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9247b3aa-e941-457c-aedb-5e7717179802 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.819281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.819947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.820234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.820512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.821255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.822628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.825507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:50.826615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.827069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.827578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.828590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:50.829571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:50.831137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa6f083-458f-4d68-881c-d9a0809a66e7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:50.843718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 465/939] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:29:50 2022] POST /identity/v3/endpoints => generated 347 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:50.844446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:50.861998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 475/940] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:50.939957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:50.941305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c0fc2-f618-404d-861e-6be92a184f2f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.941628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c0fc2-f618-404d-861e-6be92a184f2f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.941984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c0fc2-f618-404d-861e-6be92a184f2f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.944242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 476/941] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:50.951202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:50.951526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:50.951821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:50.974943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:50.976789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc83484-c4f1-4967-bef7-575f5b8c6e41 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:50.977943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:51.017278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.018115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.018396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.018672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.031988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.033561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:51.052123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:51.053081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:51.054005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:51.054005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:51.054005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:51.054005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:51.054005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 466/942] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:51.063261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.073384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 477/943] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:51.097315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.191698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.202106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1069e20-82b9-45d9-8a41-b87b02c8e50a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.233131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed3ebe-c956-453c-b84a-d8949e83bc2b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.233858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.248954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.250083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.250367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.250968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.251749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:51.252059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.254359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.270433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 467/944] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:51.276431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.285287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.286164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.286452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.286785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.287497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.288925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:51.302999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:51.304193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:51.305119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:51.309380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 478/945] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:51.310759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.412181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e51443-d11d-490c-b057-ee6b43b03b96 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.430241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0762cb70-eef7-4151-8d7f-1597733bbd7f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.449363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.450105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.450461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.450740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.458912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.460399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.476772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:51.476975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:51.477864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:51.478839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:51.479456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.479456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.479456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.479660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 468/946] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:51.483472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:51.483980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.484150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.485838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.487164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.493136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 479/947] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:51.498642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.540200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.600887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabfdb9d-9934-4b7b-bb25-21d3b33e4cb5 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.605482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfce0d3-781a-4909-b0bd-7ca98e4a108c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.640067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.641227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.641767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.642186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.642498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.643041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.643427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.643777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.644140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:51.644629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:51.644805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.645055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.646499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.646790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.657626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 480/948] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:51.658244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.667011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 469/949] 10.222.0.43 () {56 vars in 1149 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:51.675235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.725467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.768036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e3a8be-d1cc-4059-a5d2-319208fd2ef4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.786370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c421d6-0cdb-40e7-af97-a7b89530d164 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:51.787881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:51.806009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.806912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.807336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.807720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.808564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.810144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.824846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:51.825988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:51.826320 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:51.826735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:51.838242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:51.839492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 481/950] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:29:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:29:51.840028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:51.861771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:51.865349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:51.867530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:51.867530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:51.867530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:51.867530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:51.867530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 470/951] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:51.869371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:51.999270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3651b1-f6af-4836-b034-145d3748452a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:52.014544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:52.038077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:52.038761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:52.039044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:52.039354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:52.040087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:52.040398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:52.041873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:52.056879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 482/952] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:52.062337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:52.169976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402df5a8-aa80-4f9a-a365-5cafc37d9f7f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:52.205945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:52.206736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:52.207017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:52.207318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:52.214066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:52.215531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:52.229600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:52.230541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:52.231661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 471/953] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:52.238833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:52.340819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d9ffbc-53a0-41c0-9007-1ab166358847 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:52.375362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:52.376060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:52.376342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:52.376619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:52.377400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:52.377708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:52.379127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5baef7-4406-40c4-a561-f6bd79906a45 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:52.391131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 483/954] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:52.397492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:52.415546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:52.502554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dcbd9-a4a3-4dc7-acfa-8c45598e67c1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:52.537452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfe8f53-1ba8-4ffd-886f-933d193e6fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:52.538345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe8f53-1ba8-4ffd-886f-933d193e6fac None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:52.538624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe8f53-1ba8-4ffd-886f-933d193e6fac None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:52.538896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe8f53-1ba8-4ffd-886f-933d193e6fac None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/5aa970dc0f084d6f9c46cb132964c02e/roles/86faf44e636e4eb283d98058960b8ee1` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:52.541147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe8f53-1ba8-4ffd-886f-933d193e6fac None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=5aa970dc0f084d6f9c46cb132964c02e, role_id=86faf44e636e4eb283d98058960b8ee1)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:52.542678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfe8f53-1ba8-4ffd-886f-933d193e6fac None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:52.557889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 472/955] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:29:52 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/5aa970dc0f084d6f9c46cb132964c02e/roles/86faf44e636e4eb283d98058960b8ee1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:29:52.982768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18280-0f63-4338-bbe3-0b3e039a0b4a None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:52.983349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18280-0f63-4338-bbe3-0b3e039a0b4a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:52.983349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a18280-0f63-4338-bbe3-0b3e039a0b4a None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:52.984985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 484/956] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:52.990846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b5146-0b69-415c-b8b4-3302d9011fe1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:52.991314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b5146-0b69-415c-b8b4-3302d9011fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:52.991417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b5146-0b69-415c-b8b4-3302d9011fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.008930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6c7b5146-0b69-415c-b8b4-3302d9011fe1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:53.010096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6c7b5146-0b69-415c-b8b4-3302d9011fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:53.087504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 473/957] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:53.110799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18280-0f63-4338-bbe3-0b3e039a0b4a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:53.134924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb8812-9f09-4b2c-a1a2-8d2742f934cd None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.135241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb8812-9f09-4b2c-a1a2-8d2742f934cd None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.135523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb8812-9f09-4b2c-a1a2-8d2742f934cd None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.139261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 474/958] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:53.141921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.142208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.142486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.152287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:53.167658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:53.168869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:53.169423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:53.255659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 475/959] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:53.260942 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a18280-0f63-4338-bbe3-0b3e039a0b4a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:53.275251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:53.313422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:53.314228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.314515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.314796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.315485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:53.316924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:53.328849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:53.330038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:29:53.330956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:53.335063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 485/960] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:53.336349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:53.412683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc1749-4aa1-4e67-9be9-12ff2ae3cf43 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:53.439095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:53.443760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a653ae2b-4bda-47f7-a6e3-d993215ccf5f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:53.464260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:53.465021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.465216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.465490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.466209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:53.467766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:53.469140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING keystone.server.flask.common [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 24 08:29:53.477138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:53.484759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.sql.core [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 08:29:53.484759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 08:29:53.484759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [parameters: {'user_id': '396e8f5af89848b8b385ac773619f68c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 08:29:53.484759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80407) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 24 08:29:53.488847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:53.489772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.490269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.490757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.491705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] RBAC: Adding query filter params (name=compute) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:53.492223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:53.493789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:53.499270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 486/961] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:53.505051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application result.read() Jun 24 08:29:53.508015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:29:53.508991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application yield resource Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application next(self.gen) Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self.session.rollback() Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self.force_reraise() Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise self.value Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application session.commit() Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 24 08:29:53.509993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._prepare_impl() Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self.session.flush() Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._flush(objects) Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application compat.raise_( Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application flush_context.execute() Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rec.execute(self) Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 24 08:29:53.510964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 24 08:29:53.512057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application result = self._query(query) Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application conn.query(q) Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application result.read() Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 24 08:29:53.520173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [parameters: {'user_id': '396e8f5af89848b8b385ac773619f68c', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:53.522326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 24 08:29:53.525233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 476/962] 10.222.0.43 () {60 vars in 1131 bytes} [Fri Jun 24 08:29:53 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:53.528595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:53.635130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32222fbc-7a7c-40cb-b610-f77b416ed3b8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:53.641364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7760d28-da84-4d26-8d8f-1d8a5eea1cdf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:53.680568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:53.681258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.681543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.681752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:53.681893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.682550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:53.682863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:53.683417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.683899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.684308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:53.684615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.695401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 487/963] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:53 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:53.696556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:53.698569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-1a4eddd4-fb93-4748-9855-a09c09a68780 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:53.715452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:53.716498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 24 08:29:53.717469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:53.720930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 477/964] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:29:53 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:53.727016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:53.759735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:53.839177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbf6dac-a9f9-4488-a6b9-f19f1b9d7858 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:53.875603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:53.876975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:53.877096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:53.877365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:53.878164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:53.878471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:53.880624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbb758-0202-4732-abfd-e794b0d0bcc6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:53.916386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 488/965] 10.222.0.43 () {56 vars in 1101 bytes} [Fri Jun 24 08:29:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:54.035881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dbbd8-d86f-43e4-86c1-a03f59415b98 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.036383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dbbd8-d86f-43e4-86c1-a03f59415b98 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.036383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8dbbd8-d86f-43e4-86c1-a03f59415b98 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.040913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 478/966] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:54.046643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.047143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.047265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.065395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:54.066596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:54.136960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 489/967] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:54.160871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dbbd8-d86f-43e4-86c1-a03f59415b98 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.224634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:54.272619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8dbbd8-d86f-43e4-86c1-a03f59415b98 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:54.307690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:54.308495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.308825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.309101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.309756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:54.311163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:54.323292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:54.324297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:54.325149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 479/968] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:54.330800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.348526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:54.432800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98788-9818-4184-8655-e2d1973f1cea None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:54.466278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:54.466939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.467239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.467519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.468221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:54.468527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:54.470435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae55cdd-bc3f-486b-855f-07e8a15aede6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:54.479797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 490/969] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:54.480877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.581828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07bdd29-e34c-4eb8-a1d4-89efec139d02 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:54.601816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:54.608101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff9000-3c63-45f6-84b1-f392178b9d6b None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.608564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff9000-3c63-45f6-84b1-f392178b9d6b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.609059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ff9000-3c63-45f6-84b1-f392178b9d6b None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.611006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 491/970] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:54.615761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.616047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.616327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.617694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:54.618365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.618648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.618928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.619686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:54.619978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:54.621355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:54.631163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 480/971] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:54.632064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.636685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:54.638420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:54.734745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da74913-f253-46e5-8427-640f473fe967 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:54.735469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 492/972] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:54.745652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.755935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:54.772473 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:54.773129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.773411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.773685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.774741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] RBAC: Adding query filter params (interface=admin, service_id=3e6a4abab23b4da89f4d5123e33ede46, region_id=RegionOne) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:54.775055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:54.776450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:54.781781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 481/973] 10.222.0.43 () {56 vars in 1249 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/endpoints?service_id=3e6a4abab23b4da89f4d5123e33ede46&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:54.787367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.838781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:54.860231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58788807-d3dd-4a05-b97b-22b02abd690f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:54.895174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426788c6-ceaa-4869-9ed0-d2342c1ebde9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:54.896817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:54.897632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.897907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.898178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.909242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:29:54.917223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:54.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:54.919096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 493/974] 10.222.0.43 () {56 vars in 1120 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:54.924114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:54.932174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:54.932839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:54.933118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:54.933396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:54.934074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:54.935472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:54.942668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 482/975] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:29:54.994407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:55.034478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc94149-9aa1-4222-8d0a-2be4b873bbfa None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:55.073008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750b21-1658-487a-8339-f3e356df598e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:55.073799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750b21-1658-487a-8339-f3e356df598e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.074150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750b21-1658-487a-8339-f3e356df598e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.074494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47750b21-1658-487a-8339-f3e356df598e None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.075345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750b21-1658-487a-8339-f3e356df598e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:55.075723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750b21-1658-487a-8339-f3e356df598e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:55.077170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47750b21-1658-487a-8339-f3e356df598e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:55.091746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 494/976] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:29:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:55.094555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:55.203507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6badbb67-b5f7-44f6-822e-d8a183f07e1e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:55.217109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:55.239730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:55.240521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.240808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.241093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.252496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:55.253945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:55.271421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:55.272537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:55.273641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:55.273641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:55.273641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:55.273641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:55.273641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 483/977] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:55.279295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750b21-1658-487a-8339-f3e356df598e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:55.383299 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47750b21-1658-487a-8339-f3e356df598e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:55.411590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:55.418861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:55.419559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.419847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.420126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.420853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:55.421164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:55.422531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:55.435354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 495/978] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:55.441400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:55.544258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dea62f-4edf-4c31-ad2f-781711051e59 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:55.577118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:55.577917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.578197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.578469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.585868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:55.587387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:29:55.602160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:55.603172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:55.604506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 484/979] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:29:55 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:55.609041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:55.712714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda8086-307a-47a7-a9ae-9745997220b3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:55.748359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:55.749022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.749305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.749578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.750407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:55.750726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:55.752202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c36946-9c29-4e65-90e7-adebddb1c8f5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:55.769955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 496/980] 10.222.0.43 () {56 vars in 1149 bytes} [Fri Jun 24 08:29:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:55.775859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:55.802400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:55.805431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcda58-9776-461b-8d27-24540996cb93 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.806073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcda58-9776-461b-8d27-24540996cb93 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.806359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfcda58-9776-461b-8d27-24540996cb93 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.808199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 497/981] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:55.813972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.814267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.814545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.834293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:55.835993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:55.880753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a61e5e9-bb50-4182-9fd8-ace79278cd1b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:55.886139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:55.912535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 498/982] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:55.917619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:55.918265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:55.918543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:55.918815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:55.934886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:55.988939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:55.990322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.014552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:56.050339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d721e-0913-4d18-91e2-67950880ca61 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:56.091119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:56.091936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:56.092140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:56.092382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:56.093122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:56.094557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.103958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 485/983] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:29:55 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:29:56.109414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:56.111258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 499/984] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:29:55 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:29:56.221461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f287f-8739-4112-ab19-10b2457797a7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:56.246207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:56.269086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:56.269886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:56.270760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:56.270760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:56.281473 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:56.282950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:29:56.300490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:56.301421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:56.302344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:29:56.302344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:29:56.302344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:29:56.302344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:56.302344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 486/985] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:29:56.308658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:56.350713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:56.413819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c547c0-a028-43fb-a257-8fb6e5008e9a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:56.446901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:56.447589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:56.447871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:56.448148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:56.448864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:56.449170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:56.450521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.463007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 500/986] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:56.468219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:56.567105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118f4275-9b56-4d6d-be35-390a4ee91021 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:56.601690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:56.602487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:56.602766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:56.603037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:56.609748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:56.611757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:29:56.625131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:56.626660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:29:56.627964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:56.633000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 487/987] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:56.633128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:56.731904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:56.736690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd9f2b7-f3c1-475a-ab90-c31571769394 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:56.771537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:56.772478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:56.772916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:56.773345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:56.774538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:56.774998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:56.776613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5385fb88-b3fb-4a51-ad7a-bc8ebcbbb92e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.789000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 501/988] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:29:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:56.794537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:56.828062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:56.900169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524d1e60-aba9-4de5-961c-67ec02d55c7a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:56.932902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:56.933550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:56.933829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:56.934100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:56.934918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] RBAC: Adding query filter params (role.id=86faf44e636e4eb283d98058960b8ee1, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=5aa970dc0f084d6f9c46cb132964c02e) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:56.935242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:56.936663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7583de6-5e07-4227-9c79-e1b06a2535ff None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:56.943332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 488/989] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:29:56 2022] GET /identity/v3/role_assignments?user.id=5aa970dc0f084d6f9c46cb132964c02e&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=86faf44e636e4eb283d98058960b8ee1 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:57.021175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce477-6965-45af-894a-103a5b52a40f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:57.021534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce477-6965-45af-894a-103a5b52a40f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:57.021752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127ce477-6965-45af-894a-103a5b52a40f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:57.023247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 502/990] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:57.029185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:57.029479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:57.029762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:57.048672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:57.050240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:57.126399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 489/991] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:57.149908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce477-6965-45af-894a-103a5b52a40f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:57.260423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127ce477-6965-45af-894a-103a5b52a40f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:57.295001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:57.295833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:57.296117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:57.296390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:57.297039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:57.299030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:57.311091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:57.312157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:57.313008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 503/992] 10.222.0.43 () {56 vars in 1126 bytes} [Fri Jun 24 08:29:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:57.318166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:57.346179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:57.419092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f327c-f2b5-4120-8824-48e0d1a59eb9 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:57.452296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:57.452959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:57.453263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:57.453558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:57.454765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:57.455071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:57.456466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:57.461470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 490/993] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:57.466774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:57.491844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:57.566989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a80587b-7df8-499a-bca0-643c0b41d4e1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:57.603586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:57.604257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:57.604542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:57.604820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:57.605524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:57.605829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:57.607186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358c4f75-fa59-42e8-8d61-e6164e53d819 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:57.612340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 504/994] 10.222.0.43 () {56 vars in 1122 bytes} [Fri Jun 24 08:29:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:57.617679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:57.708668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:57.719572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a50c73-14ba-495a-806c-1d39823426ba None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:57.753460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479c0ac-2885-4c62-871a-55f846b13aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:57.754115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c0ac-2885-4c62-871a-55f846b13aa6 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:57.754400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c0ac-2885-4c62-871a-55f846b13aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:57.754676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479c0ac-2885-4c62-871a-55f846b13aa6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:57.755437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479c0ac-2885-4c62-871a-55f846b13aa6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:57.756922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479c0ac-2885-4c62-871a-55f846b13aa6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:57.776557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 491/995] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:29:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:29:58.182884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42151f5c-cf1e-468c-8559-c583161b5f60 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.183348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42151f5c-cf1e-468c-8559-c583161b5f60 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.183538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42151f5c-cf1e-468c-8559-c583161b5f60 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.185040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 505/996] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:58.190516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.190905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.191094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.207568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:58.208753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:58.282843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 492/997] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:58.305247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42151f5c-cf1e-468c-8559-c583161b5f60 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:58.392638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:58.409321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42151f5c-cf1e-468c-8559-c583161b5f60 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:58.443412 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:58.444134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.444416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.444690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.445339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:58.446691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:58.458283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:58.459573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:29:58.460704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:29:58.463559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 506/998] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:29:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:58.465310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:58.543097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:58.568092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dba40e9-e3f1-41fb-ac43-0e401165a20d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:58.602463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:58.603120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.603471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.603741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.604438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:58.604736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:58.606094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:58.615827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 493/999] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:58.617063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:58.723715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59768d0-0f6b-4ecd-9b98-91ca2a66b810 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:58.757212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:58.757878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.758165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.758443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.759143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:58.759479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:58.760837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:58.770564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 507/1000] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:29:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:58.772108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:58.875878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613464e2-4a8a-496a-9251-fbbf631cca49 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:58.913255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:58.913916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.914202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.914481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.915298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] RBAC: Adding query filter params (interface=public, service_id=df9f3c3d2be3460b90de11448e047890, region_id=RegionOne) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:58.915616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:58.916982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bcf2ff-0bb5-498a-9d53-be7907a8d2c2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:58.926616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 494/1001] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:29:58 2022] GET /identity/v3/endpoints?service_id=df9f3c3d2be3460b90de11448e047890&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:58.927908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:58.944686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:58.980992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd1285-2c5e-4707-b573-3d796aba9e67 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.981180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd1285-2c5e-4707-b573-3d796aba9e67 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.981463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccd1285-2c5e-4707-b573-3d796aba9e67 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.982830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 495/1002] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:29:58.987580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:58.987865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:58.988138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:58.994767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:59.005923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:59.007087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:59.041415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13dbd63-dbb8-494c-b94e-a6e549ad193e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:59.075940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:59.076604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:59.076913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:59.077167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:59.077847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:59.083386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 496/1003] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:58 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:29:59.083546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:59.089130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 508/1004] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:29:58 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:29:59.103232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:59.218867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f02b2-ab4a-425c-ad1f-6942e1e43654 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:59.256844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:59.257569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:59.257852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:59.258128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] PATH_INFO: `/v3/services/image` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:59.258789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:59.260265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:59.262101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:29:59.273109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:29:59.274600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:29:59.275709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 497/1005] 10.222.0.43 () {56 vars in 1105 bytes} [Fri Jun 24 08:29:59 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:29:59.280618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:59.361987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:59.386182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10248295-4441-439b-8a35-e33b9c3ed8c8 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:59.420417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:59.421083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:59.421367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:59.421646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:59.422347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:59.422685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:59.424071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3005c1-a109-4370-98a2-5aa1e9f003f0 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:59.429200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 509/1006] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:29:59.434886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:29:59.537653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef683c3-73be-4d13-b4a1-65e6acb4cdb4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:29:59.571099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:29:59.571806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:59.572100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:59.572379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:59.573089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] RBAC: Adding query filter params (type=image) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:29:59.573397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:29:59.574771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf2a2ca-7e48-474e-b18a-0e424b2c293f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:29:59.583448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 498/1007] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:59 2022] GET /identity/v3/services?type=image => generated 122 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:29:59.817354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c260d-3280-499a-8f5d-2984271e2418 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:59.817675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c260d-3280-499a-8f5d-2984271e2418 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:59.817915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c260d-3280-499a-8f5d-2984271e2418 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:59.819441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 510/1008] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:29:59.824572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:29:59.824863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:29:59.825140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:29:59.841756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:29:59.842930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:29:59.900421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:29:59.911946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 499/1009] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:29:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:29:59.935241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c260d-3280-499a-8f5d-2984271e2418 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:00.039182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c260d-3280-499a-8f5d-2984271e2418 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:00.073060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c742b20-38eb-4c8a-a137-38578ae85cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:00.073774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c742b20-38eb-4c8a-a137-38578ae85cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:00.074067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c742b20-38eb-4c8a-a137-38578ae85cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:00.074333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c742b20-38eb-4c8a-a137-38578ae85cb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:00.075525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c742b20-38eb-4c8a-a137-38578ae85cb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:00.076954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c742b20-38eb-4c8a-a137-38578ae85cb9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:00.083280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 511/1010] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:29:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:00.084833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:00.191952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d4cb62-ec3e-48ee-a8ae-9c1b739470ac None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:00.227727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8965c02-4b09-4a1a-9b89-fecdfbe4d48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:00.228411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8965c02-4b09-4a1a-9b89-fecdfbe4d48c None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:00.228655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8965c02-4b09-4a1a-9b89-fecdfbe4d48c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:00.228931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8965c02-4b09-4a1a-9b89-fecdfbe4d48c None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:00.229652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8965c02-4b09-4a1a-9b89-fecdfbe4d48c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:00.231089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8965c02-4b09-4a1a-9b89-fecdfbe4d48c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:00.263235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 500/1011] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:30:00 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:01.224098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6809f47-a4a8-49fb-8832-c042e30c158c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.224613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6809f47-a4a8-49fb-8832-c042e30c158c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.224704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6809f47-a4a8-49fb-8832-c042e30c158c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.226060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 512/1012] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:01.231116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.231563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.231739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.248075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:01.249252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:01.290906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:01.319986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 501/1013] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:01.343046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6809f47-a4a8-49fb-8832-c042e30c158c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:01.356435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:01.455398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6809f47-a4a8-49fb-8832-c042e30c158c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:01.498027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:01.498827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.499108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.499416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.500066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:01.501949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:01.513530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:01.514536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:01.515457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 513/1014] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:01.521163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:01.622940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d682a13-e57f-4f5b-9d13-0acf745c1c9e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:01.657052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:01.657723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.658005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.658282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.658985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:01.659317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:01.660685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd498f93-ee9e-4fc5-8b90-945eb3fb946d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:01.665592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 502/1015] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:30:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:01.671008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:01.682517 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f786c28-188c-459a-9ef6-07c836d501e6 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.682800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f786c28-188c-459a-9ef6-07c836d501e6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.683194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f786c28-188c-459a-9ef6-07c836d501e6 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.684937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 503/1016] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:30:01.690695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.690989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.691301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.712173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:01.713274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:01.744770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:01.751599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:01.779293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff96e9-8549-4bb7-9685-a8b2b20e8617 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:01.799244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 504/1017] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:30:01.814510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:01.815188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.815538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.815831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.816543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:01.816851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:01.818277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:01.828063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 514/1018] 10.222.0.43 () {56 vars in 1112 bytes} [Fri Jun 24 08:30:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:01.828245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:01.832155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:01.927488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3172d9a-deec-4e90-8a30-1899c3c9f3a2 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:01.932996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6938675e-5713-4d28-8816-c0318dc57d3d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:01.961994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914dd2f0-506e-4aae-8994-5f17fd258c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:01.962713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd2f0-506e-4aae-8994-5f17fd258c02 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.962941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd2f0-506e-4aae-8994-5f17fd258c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.963246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914dd2f0-506e-4aae-8994-5f17fd258c02 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.963967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dd2f0-506e-4aae-8994-5f17fd258c02 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:01.965345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914dd2f0-506e-4aae-8994-5f17fd258c02 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:01.967134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686b0c01-d8eb-4643-becc-5240ea4ddb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:01.967819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b0c01-d8eb-4643-becc-5240ea4ddb6c None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:01.968101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b0c01-d8eb-4643-becc-5240ea4ddb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:01.968384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686b0c01-d8eb-4643-becc-5240ea4ddb6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:01.969255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b0c01-d8eb-4643-becc-5240ea4ddb6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:01.970591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686b0c01-d8eb-4643-becc-5240ea4ddb6c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:01.979031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:01.980541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 505/1019] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:30:01 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:30:02.003248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 515/1020] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:30:01 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:02.479524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad053e4b-ae29-41b7-8efb-e4523526e80d None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:02.480081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad053e4b-ae29-41b7-8efb-e4523526e80d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:02.480081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad053e4b-ae29-41b7-8efb-e4523526e80d None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:02.481577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 506/1021] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:02.487176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:02.487584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:02.487761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:02.504165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:02.505333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:02.577313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 516/1022] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:02.602953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad053e4b-ae29-41b7-8efb-e4523526e80d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:02.631872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:02.708964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad053e4b-ae29-41b7-8efb-e4523526e80d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:02.743334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4ed248-29dd-4060-9bf4-5e7db824788f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:02.744044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ed248-29dd-4060-9bf4-5e7db824788f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:02.744322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ed248-29dd-4060-9bf4-5e7db824788f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:02.744595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4ed248-29dd-4060-9bf4-5e7db824788f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:02.745758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ed248-29dd-4060-9bf4-5e7db824788f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:02.747191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4ed248-29dd-4060-9bf4-5e7db824788f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:02.753485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 507/1023] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:02.754671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:02.855153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64ad5ae-8bde-403e-abe7-9230fb68c7e4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:02.875291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:02.891844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff6c77b-aa04-4f0f-808f-dbda6ebf30cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:02.892526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6c77b-aa04-4f0f-808f-dbda6ebf30cd None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:02.892812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6c77b-aa04-4f0f-808f-dbda6ebf30cd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:02.893092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff6c77b-aa04-4f0f-808f-dbda6ebf30cd None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:02.893799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6c77b-aa04-4f0f-808f-dbda6ebf30cd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:02.895346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff6c77b-aa04-4f0f-808f-dbda6ebf30cd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:02.921856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 517/1024] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:30:02 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:30:04.025918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29cbc1-a399-4edf-8e41-2a8e61da1c08 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.026475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29cbc1-a399-4edf-8e41-2a8e61da1c08 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.026475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d29cbc1-a399-4edf-8e41-2a8e61da1c08 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.027903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 508/1025] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:04.033588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89427797-f383-4804-963d-ec360f675e60 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.034021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89427797-f383-4804-963d-ec360f675e60 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.034235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89427797-f383-4804-963d-ec360f675e60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.051301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-89427797-f383-4804-963d-ec360f675e60 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:04.052494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-89427797-f383-4804-963d-ec360f675e60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:04.127255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 518/1026] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:04.146736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29cbc1-a399-4edf-8e41-2a8e61da1c08 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.158235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123f5a-9ee1-4293-955f-62e0bb59f5e2 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.158525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123f5a-9ee1-4293-955f-62e0bb59f5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.158805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b123f5a-9ee1-4293-955f-62e0bb59f5e2 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.160356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 519/1027] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:30:04.164979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.165279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.165559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.184326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:04.185796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:04.260839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:04.261485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 520/1028] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:30:04.269824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d29cbc1-a399-4edf-8e41-2a8e61da1c08 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.286735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.307288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.308084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.308370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.308647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] PATH_INFO: `/v3/services/image` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.309304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.310671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:04.322079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:04.323106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:04.324028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 509/1029] 10.222.0.43 () {56 vars in 1105 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:04.324875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:04.329578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.400018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371caea-d7de-47fc-bc7c-be0aa598bcd4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.435515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.436328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.436621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.436916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.444036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fdcec4-b30c-43ec-97f3-3ead41e8b509 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.448904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:04.458707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:04.459776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:04.460642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 521/1030] 10.222.0.43 () {56 vars in 1120 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:04.465854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.481351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.482018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.482299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.482578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.483310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] RBAC: Adding query filter params (name=image) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:04.483619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.485013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.494591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 510/1031] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:04.496008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.523295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:04.575407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:04.584025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe9db-0ed8-43ff-831b-6cea5a027aca None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.610722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d64f3-a0bb-4530-ba62-7863488b7e0d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.619090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.619787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.620074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.620350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.621097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:04.621402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.622768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.631597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 522/1032] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:04.636973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.646492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.647159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.647470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.647745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.648445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] RBAC: Adding query filter params (type=image) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:04.648747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.650376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.660380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 511/1033] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:04.661227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.753141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9034eda-dd95-4c37-934b-5f195bc0817d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.756893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:04.772739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cc1a57-4abb-4cfe-90b0-a9e837b6b976 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.791931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.793049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.793532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.794484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.808140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.808265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.808807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.809094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.809373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.810162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] RBAC: Adding query filter params (interface=public, service_id=dd920392791248c088c7a0713e6754c4, region_id=RegionOne) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:04.810477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.810996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.811874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.817572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 512/1034] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/endpoints?service_id=dd920392791248c088c7a0713e6754c4&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:04.822904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:04.834873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:04.835897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:04.836850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:04.836850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:04.836850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:04.836850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:04.836850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 523/1035] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:04.841496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:04.854064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:04.933266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b10f768-084b-4cc8-adac-a07302abc1aa None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.944102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a3bff-89c8-46da-a554-80aa378fb5e2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:04.969806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.970474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.970758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.971037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.971761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.973133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:04.980248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 513/1036] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:30:04.988283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:04.989273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:04.989716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:04.990147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:04.991065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:04.991570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:04.993159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa470959-2a84-4c96-9fcf-63060dd420ce None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.007715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 524/1037] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:05.013579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.032932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c967fa-67ea-43e6-a31a-056a2e96bab4 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.033260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c967fa-67ea-43e6-a31a-056a2e96bab4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.033440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c967fa-67ea-43e6-a31a-056a2e96bab4 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.034723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.035196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 525/1038] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:30:05.040881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.041182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.041448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.060051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:05.061366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:05.126329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f3f54-9923-45ea-9e65-4e0dcd6c26ab None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.133730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.140594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 526/1039] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:05.162937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.163774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.164055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.164323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.168206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.173602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.177855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:30:05.192531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:05.193501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:05.199239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 514/1040] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:05.201844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.279918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb77374-2cd5-47af-980b-ce97de0b7c5b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.295400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.319283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44733c03-7e76-4b1a-8599-25d7fdf902bb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.320272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.321077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.321354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.321642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.332201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:05.339750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:05.340702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:05.341700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 527/1041] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:05.347251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.357359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.358033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.358318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.358600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.359480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:05.359792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.361783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.384822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 515/1042] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:05.385764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.420490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.470328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989679f5-2011-4542-bbe9-954949a108a2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.495646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cac38-5fd6-4edf-97b0-3fbabdca4a10 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.504481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.505159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.505445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.505726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.506475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:05.506783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.508177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.516761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 528/1043] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:05.522048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.535623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.536278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.536564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.536841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.539531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.613973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.615766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.625700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795c5690-d7fa-4923-9e3f-dac092ed7a61 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.647526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.662034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.662910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.663190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.663492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] PATH_INFO: `/v3/users/22bf2989c2f6439392c6bbacb61eb84d` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.677251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] RBAC: Authorizing `identity:get_user(user_id=22bf2989c2f6439392c6bbacb61eb84d)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.678707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.684968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 529/1044] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/users/22bf2989c2f6439392c6bbacb61eb84d => generated 285 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:05.685945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.729404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 516/1045] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:30:05.735260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:05.769768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.790990 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46c4764-810c-4182-ba68-81fb2ca0ab31 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.825699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.826357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.826643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.826923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.841985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760533f6-d1d1-4cba-bcf7-798c96b47a63 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:05.880943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:05.881750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:05.882035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:05.882766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:05.885961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:05.895816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.897615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.910725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:05.912515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:05.920586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:05.921676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:05.922678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:05.922678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:05.922678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:05.922678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:05.922678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 517/1046] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:05.927461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:06.041706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:06.049358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 530/1047] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:30:06.050169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:06.060316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e3cbd2-7db8-4eb7-8941-be2ed581c89a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:06.110868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:06.111881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:06.112162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:06.112441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:06.113197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:06.113578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:06.115036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:06.131264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 518/1048] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:06.133473 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:06.154414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:06.183102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccb723-bd73-418f-ba42-e37f48e484e9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:06.220900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:06.221762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:06.222270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:06.222404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:06.223485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] RBAC: Authorizing `identity:get_project(project_id=14d51767bee94e3285fc724e6d0d91cf)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:06.224901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:06.226618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 531/1049] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:06 2022] GET /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:06.232619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:06.237752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4861c8f-8cec-4dbc-a80e-14543fc2b9ca None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:06.272186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:06.272906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:06.273186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:06.273458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:06.280544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:06.282001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:30:06.296908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:06.297862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:06.298792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 519/1050] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:06.312857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:06.324413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:06.362604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d438b1-4a77-444e-9afe-620cbc095f86 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:06.402523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:06.403586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:06.404093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:06.404619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:06.405807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=14d51767bee94e3285fc724e6d0d91cf, user.id=22bf2989c2f6439392c6bbacb61eb84d) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:06.406341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:06.408183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:06.414160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 532/1051] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:06 2022] GET /identity/v3/role_assignments?user.id=22bf2989c2f6439392c6bbacb61eb84d&scope.project.id=14d51767bee94e3285fc724e6d0d91cf&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:06.458130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9d746e-0ba2-41f3-bda4-8b0252ac55e9 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:06.499331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:06.500023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:06.500305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:06.501151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:06.501986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:06.502301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:06.503900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09debe2-fcf9-4fd6-ba4b-95cdeaebfd9f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:06.517587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 520/1052] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:30:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:06.524747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:06.541779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:06.645295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efb2a68-ba0e-4f20-b05c-d44b91145c2c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:06.679509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:06.680379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:06.680807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:06.681223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:06.682178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] RBAC: Adding query filter params (role.id=86faf44e636e4eb283d98058960b8ee1, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=b502d4020b494807b09e803df0b2a301) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:06.682652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:06.684254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b1292-88f8-4bb4-a727-a0842a23f43d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:06.699242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 533/1053] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:06 2022] GET /identity/v3/role_assignments?user.id=b502d4020b494807b09e803df0b2a301&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=86faf44e636e4eb283d98058960b8ee1 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:07.046237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02db2df-54b7-4e1e-bc04-4d9437efef04 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:07.046731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02db2df-54b7-4e1e-bc04-4d9437efef04 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:07.046731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02db2df-54b7-4e1e-bc04-4d9437efef04 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:07.048907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 521/1054] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:07.055826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5da65-3319-4698-947e-c6830648d19c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:07.056254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5da65-3319-4698-947e-c6830648d19c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:07.056424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b5da65-3319-4698-947e-c6830648d19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:07.072784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-21b5da65-3319-4698-947e-c6830648d19c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:07.073941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-21b5da65-3319-4698-947e-c6830648d19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:07.142272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 534/1055] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:07.167174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02db2df-54b7-4e1e-bc04-4d9437efef04 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:07.235879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:07.275580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02db2df-54b7-4e1e-bc04-4d9437efef04 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:07.308583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:07.309366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:07.309642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:07.309913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] PATH_INFO: `/v3/services/image` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:07.310557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:07.311929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:07.323125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:07.324159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 24 08:30:07.325056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:07.327530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 522/1056] 10.222.0.43 () {56 vars in 1105 bytes} [Fri Jun 24 08:30:07 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:07.330604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5da65-3319-4698-947e-c6830648d19c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:07.338634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:07.430843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b5da65-3319-4698-947e-c6830648d19c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:07.464440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:07.465099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:07.465379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:07.465655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:07.466352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:07.466654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:07.468028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:07.478817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 535/1057] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:07 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:07.479533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:07.590538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa372e-c623-46be-aaa8-52fe85273829 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:07.621166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:07.628590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:07.629264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:07.629570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:07.629853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:07.630564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:07.630872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:07.632279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3b475-d93e-4295-8613-c307696c43c0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:07.642678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 523/1058] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:07.643657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:07.717206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:07.749379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00549e17-9400-4893-a997-20c5f7d197d9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:07.785890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a55e6-091c-4bde-9668-b23b0e1a9911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:07.786618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a55e6-091c-4bde-9668-b23b0e1a9911 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:07.786904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a55e6-091c-4bde-9668-b23b0e1a9911 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:07.787297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a55e6-091c-4bde-9668-b23b0e1a9911 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:07.788057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a55e6-091c-4bde-9668-b23b0e1a9911 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:07.789422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a55e6-091c-4bde-9668-b23b0e1a9911 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:07.819305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 536/1059] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:30:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:30:08.453960 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28d1d-325e-484a-a1fa-98ca663ce255 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.454485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28d1d-325e-484a-a1fa-98ca663ce255 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.454485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de28d1d-325e-484a-a1fa-98ca663ce255 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.455973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 524/1060] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:08.461234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999b537-95b9-408f-b2bf-e3b133024b9e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.461625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999b537-95b9-408f-b2bf-e3b133024b9e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.461804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999b537-95b9-408f-b2bf-e3b133024b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.479503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2999b537-95b9-408f-b2bf-e3b133024b9e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:08.480681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2999b537-95b9-408f-b2bf-e3b133024b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:08.557098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 537/1061] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:08.579836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28d1d-325e-484a-a1fa-98ca663ce255 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:08.683028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de28d1d-325e-484a-a1fa-98ca663ce255 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:08.698021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e0fc-a328-465f-b708-e049d22de666 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.698321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e0fc-a328-465f-b708-e049d22de666 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.698714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6e0fc-a328-465f-b708-e049d22de666 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.700312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 538/1062] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:08.705629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.705854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.706132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.717169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:08.718014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.718296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.718567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.723024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:08.724229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:08.731146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:08.739154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:08.740227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 525/1063] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:08.746470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:08.775582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:08.803959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 539/1064] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:30:08.814305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:08.826251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:08.866836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54ebe7-ac9a-4661-b80d-ecdd013d9c07 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:08.905894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:08.906564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.907435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.907716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.908462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:08.908768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:08.910552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:08.924328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 526/1065] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:08.926262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:08.934846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afd4a67-236a-4698-9059-5cee45822124 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:08.976212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:08.976960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:08.977247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:08.977524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:08.988161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:08.996264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:08.997240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:08.998280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 540/1066] 10.222.0.43 () {56 vars in 1120 bytes} [Fri Jun 24 08:30:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:09.003953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.039180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb7f7ad-8e25-45ba-b9fb-b340de1bfdfd None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.049919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.058792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.081577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.082494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.082779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.083277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] PATH_INFO: `/v3/users/22bf2989c2f6439392c6bbacb61eb84d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.084671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] RBAC: Authorizing `identity:get_user(user_id=22bf2989c2f6439392c6bbacb61eb84d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.086115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.088007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 527/1067] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:08 2022] GET /identity/v3/users/22bf2989c2f6439392c6bbacb61eb84d => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:09.093336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.116767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460bf881-f76a-4b6d-aa4f-5fd9046d2f97 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.151341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.152006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.152291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.152572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.153317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:09.153623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.154978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.163193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 541/1068] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:09.169640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.211184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e274a4b-e945-4d18-ac0a-db5d0d9944e8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.245386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.246184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.246466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.246741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.247946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] RBAC: Authorizing `identity:get_project(project_id=14d51767bee94e3285fc724e6d0d91cf)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.249386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.251643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 528/1069] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:09.257316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.279530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb33f8-b756-4c7a-87bc-78f1b1fbb26e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.282718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.319412 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.320206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.320526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.320804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.321202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.332889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.334327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:09.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:09.352004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:09.353195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:09.353195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:09.353195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:09.353195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:09.353195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 542/1070] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:09.357777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.377652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f455a-0b11-40e2-bca5-f75480c1dce6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.414255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.415178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.415502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.415782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf/users/22bf2989c2f6439392c6bbacb61eb84d/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.417858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] RBAC: Authorizing `identity:create_grant(project_id=14d51767bee94e3285fc724e6d0d91cf, user_id=22bf2989c2f6439392c6bbacb61eb84d, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.419439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.443256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 529/1071] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:30:09 2022] PUT /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf/users/22bf2989c2f6439392c6bbacb61eb84d/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:30:09.472387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56756f9-0a2e-4f24-8ea8-e252fda1c9c3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.513602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.514183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.514463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.514735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.515512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:09.515817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.517256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.531143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 543/1072] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:09.536914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.656979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fdad07-39e0-4ada-a06e-b826e0137400 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.669918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.693714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.694512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.694795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.695072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.702791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.704307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-8c49fe91-9ed5-4515-8e42-5f02af84b582 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:30:09.718467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:09.720000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:09.721016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 530/1073] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:09.728501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.748020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.833559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815556f3-f7c3-400f-9c10-23439e6684bd None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:09.868764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:09.869435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.869716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.869983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.870811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:09.871117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:09.872884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:09.874871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b48f8-f8f8-4db3-8c11-e958c1db9044 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.875352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b48f8-f8f8-4db3-8c11-e958c1db9044 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.875641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4b48f8-f8f8-4db3-8c11-e958c1db9044 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.877333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 531/1074] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:09.882553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:09.883053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:09.883389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:09.892329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 544/1075] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:09.897890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:09.901709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:09.902905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:09.977622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:09.987117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 532/1076] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:30:10.007391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d93a3-19e2-438c-813c-fdeffbf85c3a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.009239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.046289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.047089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.047473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.048235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.059131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.060576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:10.077562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:10.078489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:10.079971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:10.079971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:10.079971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:10.079971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:10.079971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 545/1077] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:10.084821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.127247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47fc57-0737-447f-9300-1ef09e5482fe None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.129577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:10.166320 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.166988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.167300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.167578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.198737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d11f3-cf20-42cc-bb0b-37d3cb0dc55f None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.229791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:10.237652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.239234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.239839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.240502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.240785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.241061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.241782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:10.242082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.243955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.256861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 546/1078] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:10.262219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.354742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 533/1079] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:30:10.361270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.363272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:10.374417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3922d43a-2ee1-44e7-9966-ca05f469460a None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.409743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.410537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.410817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.411090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.417970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.419433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:30:10.435165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:10.436219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:10.437660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 547/1080] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:10.442551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.473132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881fc3c-ba1e-4928-8cf5-cf3906e061e1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.494731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:10.511323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.512126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.512411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.512690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.522796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.524292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:10.542994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:10.544635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:10.545763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:10.545763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:10.545763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:10.545763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:10.551395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 534/1081] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:10.552576 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.559991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ebd93-493c-480d-873e-22d56df063a3 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.602611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.603311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.603597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.603874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.605140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:10.605404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.607494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.620414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 548/1082] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:30:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:10.625152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:10.626032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:10.679967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46022a8-6391-42be-bae7-6edb666efa93 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.716368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.717148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.717492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.717895 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.718670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:10.718982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.721007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ce6258-b210-4cd2-bc4c-d859e582c8e5 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.726999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29ab4c9-1ecf-4059-a6e1-34b614391aa1 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:10.739247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 535/1083] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:10.739453 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:10.769409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503f11ba-56b3-406c-8060-d9a7b660775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:10.770428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f11ba-56b3-406c-8060-d9a7b660775b None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:10.770781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f11ba-56b3-406c-8060-d9a7b660775b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:10.771156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503f11ba-56b3-406c-8060-d9a7b660775b None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/b502d4020b494807b09e803df0b2a301/roles/86faf44e636e4eb283d98058960b8ee1` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:10.773564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f11ba-56b3-406c-8060-d9a7b660775b None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=b502d4020b494807b09e803df0b2a301, role_id=86faf44e636e4eb283d98058960b8ee1)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:10.775493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503f11ba-56b3-406c-8060-d9a7b660775b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:10.792874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 549/1084] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:30:10 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/b502d4020b494807b09e803df0b2a301/roles/86faf44e636e4eb283d98058960b8ee1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:30:11.492080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdaf37-7f9c-458c-a34d-e721e751159f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:11.492317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdaf37-7f9c-458c-a34d-e721e751159f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:11.492644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbdaf37-7f9c-458c-a34d-e721e751159f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:11.493939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 536/1085] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:11.499042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:11.499531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:11.499709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:11.517260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:11.518817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:11.585672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 550/1086] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:11.608450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdaf37-7f9c-458c-a34d-e721e751159f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:11.726523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbdaf37-7f9c-458c-a34d-e721e751159f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:11.741459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:11.762237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:11.762962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:11.763288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:11.763567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:11.773802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:11.781650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:11.782771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:11.783835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:11.787694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 537/1087] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:11.789005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:11.888338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:11.897510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d9f9f3-f234-4d5b-97c5-6c17123c81cb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:11.936978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:11.937640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:11.937986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:11.938768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:11.939538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:11.939845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:11.941190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:11.952077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 551/1088] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:11.957596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:12.055691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376f157-a05f-463d-9083-529d402e4fb4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:12.088261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:12.089120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:12.089399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:12.089672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] PATH_INFO: `/v3/users/22bf2989c2f6439392c6bbacb61eb84d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:12.090835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] RBAC: Authorizing `identity:get_user(user_id=22bf2989c2f6439392c6bbacb61eb84d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:12.092298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:12.093894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 538/1089] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:11 2022] GET /identity/v3/users/22bf2989c2f6439392c6bbacb61eb84d => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:12.100000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:12.204152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80765ca1-78e1-4072-8d7d-398bab541112 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:12.237709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:12.238373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:12.238655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:12.238928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:12.266798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:12.309473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:12.310866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:12.413780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 552/1090] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:30:12.419556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:12.462489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:12.522185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ee09f0-6d36-41f3-a893-64acbfc8ed36 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:12.555635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792b57a7-d48f-45fe-b4f9-99876c5df0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:12.556502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b57a7-d48f-45fe-b4f9-99876c5df0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:12.556785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b57a7-d48f-45fe-b4f9-99876c5df0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:12.557061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792b57a7-d48f-45fe-b4f9-99876c5df0b6 None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:12.558271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b57a7-d48f-45fe-b4f9-99876c5df0b6 None admin] RBAC: Authorizing `identity:get_project(project_id=14d51767bee94e3285fc724e6d0d91cf)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:12.559799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792b57a7-d48f-45fe-b4f9-99876c5df0b6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:12.561451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 539/1091] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:12 2022] GET /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:12.567417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:12.626761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:12.667101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a5f1b4-c48d-4a51-89f6-2eb844148feb None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:12.702357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:12.703007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:12.703319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:12.703628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:12.704448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=14d51767bee94e3285fc724e6d0d91cf, user.id=22bf2989c2f6439392c6bbacb61eb84d) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:12.704756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:12.706196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f979f0-1239-48c7-b889-8d26d802695c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:12.712106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 553/1092] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:12 2022] GET /identity/v3/role_assignments?user.id=22bf2989c2f6439392c6bbacb61eb84d&scope.project.id=14d51767bee94e3285fc724e6d0d91cf&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:12.832210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5852193-3c6b-48ef-8f43-82c7675c2f70 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:12.832719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5852193-3c6b-48ef-8f43-82c7675c2f70 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:12.832719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5852193-3c6b-48ef-8f43-82c7675c2f70 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:12.834077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 540/1093] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:12.839698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:12.839995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:12.840280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:12.857917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:12.859116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:12.930470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 554/1094] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:12.954087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5852193-3c6b-48ef-8f43-82c7675c2f70 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:13.054483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5852193-3c6b-48ef-8f43-82c7675c2f70 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:13.065039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:13.088972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:13.089710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:13.089993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:13.090270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:13.100867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:13.108938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:13.109988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:13.110850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 541/1095] 10.222.0.43 () {56 vars in 1120 bytes} [Fri Jun 24 08:30:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:13.117771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:13.169459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:13.220259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e309d7-b772-43b7-bba3-b46f36a3fa39 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:13.254198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:13.254865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:13.255171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:13.255476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:13.256218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:13.256524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:13.257875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:13.265862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 555/1096] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:30:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:13.271956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:13.371681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d356fc6-ddb5-4785-a39c-9f414eb563c8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:13.405643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:13.406433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:13.407336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:13.407623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:13.418174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:13.419674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:13.436425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:13.437526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:13.439037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:13.439037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:13.439037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:13.439037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:13.442718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 542/1097] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:13.443919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:13.541938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f071c07f-aace-49c6-92bc-067b412a46d9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:13.555074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:13.575183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:13.575871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:13.576151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:13.576425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:13.577135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:13.577441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:13.579352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:13.590556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 556/1098] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:13.596103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:13.620815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:13.695162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807da65-4627-4d37-a689-7d1fe57699a4 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:13.727410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:13.728188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:13.728468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:13.728742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:13.735911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:13.737345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:30:13.751418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:13.752414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:13.753404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 543/1099] 10.222.0.43 () {56 vars in 1093 bytes} [Fri Jun 24 08:30:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:13.758305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:13.858316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78c756-024e-44e1-b1e3-956d5f0c1da9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:13.900398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:13.901074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:13.901356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:13.901638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:13.902478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:13.902789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:13.904239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:13.921167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 557/1100] 10.222.0.43 () {56 vars in 1133 bytes} [Fri Jun 24 08:30:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:13.927297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.008403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:14.026613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee4b6bf-a641-4867-8413-25a859569122 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.058784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:14.059453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.059743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.060022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.126910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:14.128528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:14.211860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:14.231300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 544/1101] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:14.237593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.267887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:14.338351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968627d0-24d7-4642-b3f1-cdca4f7714d2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.372321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:14.373108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.373389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.373663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.383810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:14.385243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:14.402861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:14.403871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:14.405373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:30:14.405373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:14.405373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:30:14.405373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:14.405373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 558/1102] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:30:14.410152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.506709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6d604-a2d8-4414-8076-9ebae1fbee8d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.540283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:14.540876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.541159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.541432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.542147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:14.542451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:14.543864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:14.555677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 545/1103] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:14.560839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.646077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:14.662684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d04f3-9e7a-4691-96ea-5cbcd240a75c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.695616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:14.696408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.696682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.696958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.703426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:14.705366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4fa571db-38b6-4b7e-a0b6-7e6f9f06cc19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:30:14.718803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:14.720372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:14.721451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 559/1104] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:30:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:14.726332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.764666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adcb4c-a9b6-400c-b3e8-4aeec77de78a None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.764957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adcb4c-a9b6-400c-b3e8-4aeec77de78a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.765241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adcb4c-a9b6-400c-b3e8-4aeec77de78a None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.766686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 560/1105] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:14.769186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:14.771792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.772087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.772369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.789086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:14.790261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:14.833442 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f43d65-cc60-4b26-b9a2-1e4c3f4762f6 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.858149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 561/1106] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:30:14.866694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:14.867430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:14.867708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:14.868003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:14.868780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:14.869084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:14.870504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:14.883528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 546/1107] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:30:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:14.884346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.889171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:14.956979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:14.981459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8615398-c370-4b7e-8cbe-7e03c34ebb95 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.988671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de06ac-ba04-4a29-b4e6-f006b5196a98 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:14.990192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:15.015255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:15.016078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:15.016362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:15.016638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:15.017817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:15.019248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:15.020870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 562/1108] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:15.025300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:15.025951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:15.026232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:15.026504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:15.026604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:15.027359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] RBAC: Adding query filter params (role.id=86faf44e636e4eb283d98058960b8ee1, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=b502d4020b494807b09e803df0b2a301) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:15.027679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:15.029112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e036cf48-b021-4905-b964-cfd377c10d26 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:15.036058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 547/1109] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:14 2022] GET /identity/v3/role_assignments?user.id=b502d4020b494807b09e803df0b2a301&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=86faf44e636e4eb283d98058960b8ee1 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:15.133776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587468f9-a483-441d-8b8b-3d3710595d29 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:15.170260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cda92-4471-4882-a8e0-d1132e3f0284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:15.170944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cda92-4471-4882-a8e0-d1132e3f0284 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:15.171242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cda92-4471-4882-a8e0-d1132e3f0284 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:15.171522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cda92-4471-4882-a8e0-d1132e3f0284 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:15.172390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cda92-4471-4882-a8e0-d1132e3f0284 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:15.174312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cda92-4471-4882-a8e0-d1132e3f0284 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:15.190637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:15.197397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 563/1110] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:30:15 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:30:17.284846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2429126-423b-4442-b04c-0eb01569bd40 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:17.285385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2429126-423b-4442-b04c-0eb01569bd40 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:17.285385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2429126-423b-4442-b04c-0eb01569bd40 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:17.286770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 548/1111] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:17.291921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:17.292325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:17.292485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:17.310029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:17.311191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:17.374118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 564/1112] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:17.398128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2429126-423b-4442-b04c-0eb01569bd40 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:17.494619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2429126-423b-4442-b04c-0eb01569bd40 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:17.527741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:17.528594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:17.528879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:17.529155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] PATH_INFO: `/v3/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:17.530650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] RBAC: Authorizing `identity:get_role(role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:17.532107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:17.533498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 549/1113] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:17 2022] GET /identity/v3/roles/f4b3603604994326b650f99b4db1e7a0 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:17.539224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:17.637291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece797b-ab12-4c19-af0f-b2b4d741e53d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:17.664022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:17.671747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:17.673112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:17.673391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:17.673659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] PATH_INFO: `/v3/users/7599633a8f2740f9910749b462143e26` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:17.686373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] RBAC: Authorizing `identity:get_user(user_id=7599633a8f2740f9910749b462143e26)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:17.688474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:17.690129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 565/1114] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:17 2022] GET /identity/v3/users/7599633a8f2740f9910749b462143e26 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:17.695861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:17.728433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:17.793133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab5ed6-8f2c-4a61-bd1b-858a2aab378e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:17.826696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:17.827373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:17.827661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:17.827941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:17.896874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:17.898336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:17.932592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:17.997144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 550/1115] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:18.002918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:18.099544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12b0fc-9620-4c1a-b81f-156976355d30 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:18.133513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fece89-a82c-481f-8775-3f2c15e5f9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:18.134368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fece89-a82c-481f-8775-3f2c15e5f9fb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:18.134653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fece89-a82c-481f-8775-3f2c15e5f9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:18.134929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fece89-a82c-481f-8775-3f2c15e5f9fb None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:18.136129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fece89-a82c-481f-8775-3f2c15e5f9fb None admin] RBAC: Authorizing `identity:get_project(project_id=14d51767bee94e3285fc724e6d0d91cf)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:18.137554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fece89-a82c-481f-8775-3f2c15e5f9fb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:18.139259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 566/1116] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:18 2022] GET /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:18.145447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:18.242749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18040e69-bc9a-46b6-8856-d485ecf8b392 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:18.274983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:18.275649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:18.275931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:18.276207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:18.277017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] RBAC: Adding query filter params (role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=14d51767bee94e3285fc724e6d0d91cf, user.id=7599633a8f2740f9910749b462143e26) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:18.277323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:18.278740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022b910a-047a-4815-ad2f-0707bf9d74ab None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:18.280504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:18.285453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 551/1117] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:18 2022] GET /identity/v3/role_assignments?user.id=7599633a8f2740f9910749b462143e26&scope.project.id=14d51767bee94e3285fc724e6d0d91cf&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:20.306119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541620-c027-4ccd-87e0-d9f29815fad1 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:20.306666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541620-c027-4ccd-87e0-d9f29815fad1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:20.306666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28541620-c027-4ccd-87e0-d9f29815fad1 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:20.308082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 567/1118] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:20.313707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:20.314126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:20.314260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:20.331868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:20.333054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:20.397540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 552/1119] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:20.420502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541620-c027-4ccd-87e0-d9f29815fad1 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:20.477753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:20.521980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28541620-c027-4ccd-87e0-d9f29815fad1 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:20.554474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:20.555361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:20.555648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:20.555927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] PATH_INFO: `/v3/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:20.557388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] RBAC: Authorizing `identity:get_role(role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:20.558752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:20.560176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 568/1120] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:20 2022] GET /identity/v3/roles/f4b3603604994326b650f99b4db1e7a0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:20.566553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:20.662101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecbdacb-00f5-4aac-9544-b074241afb7d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:20.688680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:20.696382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:20.697243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:20.697524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:20.697797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] PATH_INFO: `/v3/users/7599633a8f2740f9910749b462143e26` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:20.698967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] RBAC: Authorizing `identity:get_user(user_id=7599633a8f2740f9910749b462143e26)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:20.700446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:20.702047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 553/1121] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:20 2022] GET /identity/v3/users/7599633a8f2740f9910749b462143e26 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:20.707950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:20.805106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5c4e7-549b-4e82-b255-27895efad0bd None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:20.827337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:20.838739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac84bfbd-f52d-416d-87ac-0faaa08612f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:20.839621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84bfbd-f52d-416d-87ac-0faaa08612f1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:20.839910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84bfbd-f52d-416d-87ac-0faaa08612f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:20.840190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac84bfbd-f52d-416d-87ac-0faaa08612f1 None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:20.841362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84bfbd-f52d-416d-87ac-0faaa08612f1 None admin] RBAC: Authorizing `identity:get_project(project_id=14d51767bee94e3285fc724e6d0d91cf)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:20.842777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac84bfbd-f52d-416d-87ac-0faaa08612f1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:20.844532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 569/1122] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:20 2022] GET /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:20.850334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:20.953594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab46948-1b96-43bc-9455-6ffd24636214 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:20.986930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c589b601-2da0-4059-87a3-c9e46731b933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:20.987837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b601-2da0-4059-87a3-c9e46731b933 None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:20.988122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b601-2da0-4059-87a3-c9e46731b933 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:20.988398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c589b601-2da0-4059-87a3-c9e46731b933 None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf/users/7599633a8f2740f9910749b462143e26/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:20.990562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589b601-2da0-4059-87a3-c9e46731b933 None admin] RBAC: Authorizing `identity:create_grant(project_id=14d51767bee94e3285fc724e6d0d91cf, user_id=7599633a8f2740f9910749b462143e26, role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:20.992031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c589b601-2da0-4059-87a3-c9e46731b933 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:21.004586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 554/1123] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:30:20 2022] PUT /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf/users/7599633a8f2740f9910749b462143e26/roles/f4b3603604994326b650f99b4db1e7a0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:30:22.982033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a8468-7253-4142-9543-f9089ac179dc None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:22.982618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a8468-7253-4142-9543-f9089ac179dc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:22.982618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a8468-7253-4142-9543-f9089ac179dc None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:22.983999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 570/1124] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:22.989273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:22.989669 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:22.989832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:23.005975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:23.007147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:23.056383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:23.073180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 555/1125] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:23.097577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a8468-7253-4142-9543-f9089ac179dc None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:23.197857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a8468-7253-4142-9543-f9089ac179dc None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:23.231200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:23.232127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:23.232369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:23.232644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] PATH_INFO: `/v3/roles/f4b3603604994326b650f99b4db1e7a0` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:23.234108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] RBAC: Authorizing `identity:get_role(role_id=f4b3603604994326b650f99b4db1e7a0)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:23.235522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:23.236894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 571/1126] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:23 2022] GET /identity/v3/roles/f4b3603604994326b650f99b4db1e7a0 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:23.242647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:23.338380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53881d-bd18-4e82-8c58-85caaeb747b4 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:23.370282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:23.371144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:23.371452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:23.371727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] PATH_INFO: `/v3/users/7599633a8f2740f9910749b462143e26` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:23.372883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] RBAC: Authorizing `identity:get_user(user_id=7599633a8f2740f9910749b462143e26)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:23.374304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:23.376428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 556/1127] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:23 2022] GET /identity/v3/users/7599633a8f2740f9910749b462143e26 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:23.382263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:23.439887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:23.480615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ee449f-651a-45d5-8cb8-d4c81639c34b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:23.512980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:23.513567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:23.513847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:23.514121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:23.579072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:23.580538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:23.634411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:23.675533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 572/1128] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:23.681411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:23.734092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:23.777579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceeee71-9885-488b-bfd1-3de5e6167557 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:23.809672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efc896-59eb-4ff9-b14e-d3eb77dff1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:23.810508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efc896-59eb-4ff9-b14e-d3eb77dff1ac None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:23.810784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efc896-59eb-4ff9-b14e-d3eb77dff1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:23.811053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efc896-59eb-4ff9-b14e-d3eb77dff1ac None admin] PATH_INFO: `/v3/projects/14d51767bee94e3285fc724e6d0d91cf` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:23.812234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efc896-59eb-4ff9-b14e-d3eb77dff1ac None admin] RBAC: Authorizing `identity:get_project(project_id=14d51767bee94e3285fc724e6d0d91cf)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:23.814269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efc896-59eb-4ff9-b14e-d3eb77dff1ac None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:23.815968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 557/1129] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:23 2022] GET /identity/v3/projects/14d51767bee94e3285fc724e6d0d91cf => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:23.821952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:23.916354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814dcbde-c1f2-4310-b02e-b3c312541526 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:23.956742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:23.957394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:23.957676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:23.957954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:23.958776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] RBAC: Adding query filter params (role.id=f4b3603604994326b650f99b4db1e7a0, scope.project.id=14d51767bee94e3285fc724e6d0d91cf, user.id=7599633a8f2740f9910749b462143e26) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:23.959096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:23.960543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbc4ee-9362-4c6e-ba3f-c0c4a7702a8a None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:23.966282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 573/1130] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:23 2022] GET /identity/v3/role_assignments?user.id=7599633a8f2740f9910749b462143e26&scope.project.id=14d51767bee94e3285fc724e6d0d91cf&role.id=f4b3603604994326b650f99b4db1e7a0 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:26.001585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb2853-630d-4ec8-9a39-a4985210f813 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:26.002112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb2853-630d-4ec8-9a39-a4985210f813 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:26.002112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bb2853-630d-4ec8-9a39-a4985210f813 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:26.003533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 558/1131] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:26.008590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:26.009043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:26.009139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:26.025656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:26.026811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:26.042537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:26.091004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 574/1132] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:26.114982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb2853-630d-4ec8-9a39-a4985210f813 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:26.215274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bb2853-630d-4ec8-9a39-a4985210f813 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:26.240585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:26.252315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c980db-119c-4ff1-9495-ab60fbfb574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:26.253044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c980db-119c-4ff1-9495-ab60fbfb574b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:26.253323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c980db-119c-4ff1-9495-ab60fbfb574b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:26.253596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c980db-119c-4ff1-9495-ab60fbfb574b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:26.254756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c980db-119c-4ff1-9495-ab60fbfb574b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:26.256221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c980db-119c-4ff1-9495-ab60fbfb574b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:26.257830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 559/1133] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:26.263771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:26.359026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01fca76-028e-4607-9498-c41e56d4a5d9 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:26.393172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f251fe7-00cc-44ff-862c-9faa068838fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:26.393825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f251fe7-00cc-44ff-862c-9faa068838fc None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:26.394108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f251fe7-00cc-44ff-862c-9faa068838fc None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:26.394384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f251fe7-00cc-44ff-862c-9faa068838fc None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:26.395083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f251fe7-00cc-44ff-862c-9faa068838fc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:26.396536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f251fe7-00cc-44ff-862c-9faa068838fc None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:26.420892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 575/1134] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:30:26 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:28.462772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f0695-7e1f-408d-8c03-ceca165d5efa None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:28.463301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f0695-7e1f-408d-8c03-ceca165d5efa None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:28.463301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f0695-7e1f-408d-8c03-ceca165d5efa None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:28.464687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 560/1135] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:28.469986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:28.470423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:28.470541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:28.476787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:28.488859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:28.490051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:28.558068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 576/1136] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:28.584038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f0695-7e1f-408d-8c03-ceca165d5efa None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:28.686077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f0695-7e1f-408d-8c03-ceca165d5efa None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:28.721022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e06333f-39e3-41ec-b320-6ebab77da68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:28.721754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06333f-39e3-41ec-b320-6ebab77da68d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:28.722045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06333f-39e3-41ec-b320-6ebab77da68d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:28.722325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e06333f-39e3-41ec-b320-6ebab77da68d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:28.723541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06333f-39e3-41ec-b320-6ebab77da68d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:28.724986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e06333f-39e3-41ec-b320-6ebab77da68d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:28.726621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 561/1137] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:30:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:28.732391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:28.831947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b897c8-b290-471a-ab92-fd7f355e980e None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:28.865993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b20d01-79b8-4af9-a6d7-0073fcb6f8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:28.866666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b20d01-79b8-4af9-a6d7-0073fcb6f8c9 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:28.866945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b20d01-79b8-4af9-a6d7-0073fcb6f8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:28.867289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b20d01-79b8-4af9-a6d7-0073fcb6f8c9 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:28.868032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b20d01-79b8-4af9-a6d7-0073fcb6f8c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:28.869553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b20d01-79b8-4af9-a6d7-0073fcb6f8c9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:28.876898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:28.890952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 577/1138] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:30:28 2022] POST /identity/v3/users => generated 286 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:30.937474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea1d02-0da9-484f-9346-ce050c8f5e80 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:30.937995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea1d02-0da9-484f-9346-ce050c8f5e80 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:30.938113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea1d02-0da9-484f-9346-ce050c8f5e80 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:30.939475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 562/1139] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:30.944897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:30.945333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:30.945433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:30.962274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:30.963510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:31.034765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 578/1140] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:31.056519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea1d02-0da9-484f-9346-ce050c8f5e80 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:31.117854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:31.160548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea1d02-0da9-484f-9346-ce050c8f5e80 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:31.194787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:31.195626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:31.195915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:31.196195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:31.207457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:31.214989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:31.216233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:31.217369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:31.219502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 563/1141] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:31.222878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:31.321219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b40919-8f71-4664-821a-e79280b67ef3 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:31.342258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:31.357988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:31.358663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:31.358945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:31.359266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:31.360027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:31.360332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:31.361689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:31.369926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 579/1142] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:31.376022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:31.479968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd669-97c7-43e4-8198-5b168336808e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:31.496659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:31.513931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:31.514812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:31.515092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:31.515396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] PATH_INFO: `/v3/users/acd469f7110f46108444e99547c9c0b7` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:31.529871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] RBAC: Authorizing `identity:get_user(user_id=acd469f7110f46108444e99547c9c0b7)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:31.531384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:31.533151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 564/1143] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:31 2022] GET /identity/v3/users/acd469f7110f46108444e99547c9c0b7 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:31.539226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:31.641315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578181a-3f97-443a-bdfb-cbf86a174787 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:31.675263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:31.675871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:31.676159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:31.676433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:31.725713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:31.744866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:31.746254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:31.847140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 580/1144] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:31.853036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:31.955173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca501005-52da-4c00-9b44-7c15d48018ae None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:31.989803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e274d-7e11-4716-ac87-9d8284d353d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:31.991457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e274d-7e11-4716-ac87-9d8284d353d6 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:31.991760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e274d-7e11-4716-ac87-9d8284d353d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:31.992284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e274d-7e11-4716-ac87-9d8284d353d6 None admin] PATH_INFO: `/v3/projects/9e3e6e52c95e46d78cd08b915e68395e` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:31.993350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e274d-7e11-4716-ac87-9d8284d353d6 None admin] RBAC: Authorizing `identity:get_project(project_id=9e3e6e52c95e46d78cd08b915e68395e)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:31.994921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e274d-7e11-4716-ac87-9d8284d353d6 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:31.996626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 565/1145] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:31 2022] GET /identity/v3/projects/9e3e6e52c95e46d78cd08b915e68395e => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:32.003544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:32.085235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:32.103625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a46a0c-dda1-4f4c-ba9e-c60d1102bcbf None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:32.138422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:32.139076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:32.139392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:32.139671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:32.140493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=9e3e6e52c95e46d78cd08b915e68395e, user.id=acd469f7110f46108444e99547c9c0b7) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:32.140832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:32.142252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196dd7cf-bbaa-4a7c-8c73-1373bbfbbac1 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:32.147765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 581/1146] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:32 2022] GET /identity/v3/role_assignments?user.id=acd469f7110f46108444e99547c9c0b7&scope.project.id=9e3e6e52c95e46d78cd08b915e68395e&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:34.139244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543489ed-d33b-47cb-8de6-72a45bc7cf98 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.139766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543489ed-d33b-47cb-8de6-72a45bc7cf98 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.139854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543489ed-d33b-47cb-8de6-72a45bc7cf98 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.141862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 566/1147] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:34.146767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.147146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.147351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.164616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:34.165793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:34.233545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 582/1148] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:34.255694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543489ed-d33b-47cb-8de6-72a45bc7cf98 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:34.302342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:34.357406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543489ed-d33b-47cb-8de6-72a45bc7cf98 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:34.391156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:34.391915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.392198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.392475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.402451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:34.410542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:34.411650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:34.412676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 567/1149] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:34.417819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:34.491451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:34.514500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c716f438-8161-402c-9632-2fc749646fa5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:34.547555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:34.548217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.548503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.548784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.549533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:34.549845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:34.551203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:34.559156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 583/1150] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:34.565310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:34.668279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4dfc-152b-4d5c-aa15-8c24e1d8ba7f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:34.670515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:34.701498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:34.702359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.702643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.702918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] PATH_INFO: `/v3/users/acd469f7110f46108444e99547c9c0b7` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.704733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] RBAC: Authorizing `identity:get_user(user_id=acd469f7110f46108444e99547c9c0b7)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:34.706165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:34.707820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 568/1151] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:34 2022] GET /identity/v3/users/acd469f7110f46108444e99547c9c0b7 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:34.713607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:34.808977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e19d91f-794e-4dc6-873d-e92f56702414 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:34.841266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054bc55b-358d-46af-b2a7-9d409394e466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:34.842113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc55b-358d-46af-b2a7-9d409394e466 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.842397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc55b-358d-46af-b2a7-9d409394e466 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.842670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054bc55b-358d-46af-b2a7-9d409394e466 None admin] PATH_INFO: `/v3/projects/9e3e6e52c95e46d78cd08b915e68395e` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.843897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bc55b-358d-46af-b2a7-9d409394e466 None admin] RBAC: Authorizing `identity:get_project(project_id=9e3e6e52c95e46d78cd08b915e68395e)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:34.845307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054bc55b-358d-46af-b2a7-9d409394e466 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:34.846964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 584/1152] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:34 2022] GET /identity/v3/projects/9e3e6e52c95e46d78cd08b915e68395e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:34.852643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:34.948978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fc21b-867d-4e25-a346-1bc59a9ead4f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:34.980248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17da96-a36f-40b8-8652-50c812d492fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:34.981117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17da96-a36f-40b8-8652-50c812d492fb None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:34.981397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17da96-a36f-40b8-8652-50c812d492fb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:34.981669 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17da96-a36f-40b8-8652-50c812d492fb None admin] PATH_INFO: `/v3/projects/9e3e6e52c95e46d78cd08b915e68395e/users/acd469f7110f46108444e99547c9c0b7/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:34.983598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17da96-a36f-40b8-8652-50c812d492fb None admin] RBAC: Authorizing `identity:create_grant(project_id=9e3e6e52c95e46d78cd08b915e68395e, user_id=acd469f7110f46108444e99547c9c0b7, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:34.985508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17da96-a36f-40b8-8652-50c812d492fb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:34.998240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 569/1153] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:30:34 2022] PUT /identity/v3/projects/9e3e6e52c95e46d78cd08b915e68395e/users/acd469f7110f46108444e99547c9c0b7/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:30:36.996031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885a9d-1f37-4a8c-807a-61472fbc8327 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:36.996583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885a9d-1f37-4a8c-807a-61472fbc8327 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:36.996583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd885a9d-1f37-4a8c-807a-61472fbc8327 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:36.997957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 585/1154] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:37.003103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:37.003545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:37.003702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:37.020041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:37.021220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:37.046658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:37.087494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 570/1155] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:37.111386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885a9d-1f37-4a8c-807a-61472fbc8327 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:37.120452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:37.209947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd885a9d-1f37-4a8c-807a-61472fbc8327 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:37.242333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:37.243108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:37.243418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:37.243696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:37.254548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:37.261912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:37.262825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:37.263826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 586/1156] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:37.269627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:37.364941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d377693-179f-4439-a620-02e7ac4ab1a2 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:37.397305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:37.397959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:37.398240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:37.398514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:37.399277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:37.399845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:37.401216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:37.409459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 571/1157] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:37.414955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:37.477700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:37.510954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cd9e5-d3d0-4fc6-b8d5-3d7b12d68153 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:37.543352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:37.544198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:37.544478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:37.544750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] PATH_INFO: `/v3/users/acd469f7110f46108444e99547c9c0b7` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:37.545901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] RBAC: Authorizing `identity:get_user(user_id=acd469f7110f46108444e99547c9c0b7)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:37.547372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:37.548952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 587/1158] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:37 2022] GET /identity/v3/users/acd469f7110f46108444e99547c9c0b7 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:37.554614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:37.580050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:37.653184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87f73e-3c25-4bcb-9203-f009b09beb6e None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:37.684381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:37.685018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:37.685299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:37.685575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:37.752660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:37.754093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:37.777827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:37.849864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 572/1159] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:37.855708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:37.951624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9deed0-5be2-423b-abf5-1480b368c898 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:37.983464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec64468-d761-4062-a5ca-da31e73e8f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:37.984824 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec64468-d761-4062-a5ca-da31e73e8f0c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:37.985103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec64468-d761-4062-a5ca-da31e73e8f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:37.985381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec64468-d761-4062-a5ca-da31e73e8f0c None admin] PATH_INFO: `/v3/projects/9e3e6e52c95e46d78cd08b915e68395e` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:37.986527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec64468-d761-4062-a5ca-da31e73e8f0c None admin] RBAC: Authorizing `identity:get_project(project_id=9e3e6e52c95e46d78cd08b915e68395e)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:37.987961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec64468-d761-4062-a5ca-da31e73e8f0c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:37.989602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 588/1160] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:37 2022] GET /identity/v3/projects/9e3e6e52c95e46d78cd08b915e68395e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:37.995523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:38.089661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286482e-7239-4c99-bf34-82c7a855ce13 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:38.112100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:38.122706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:38.123374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:38.123657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:38.123928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:38.124744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=9e3e6e52c95e46d78cd08b915e68395e, user.id=acd469f7110f46108444e99547c9c0b7) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:38.125594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:38.127054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885a6259-605a-4477-943b-6868d42ade31 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:38.132716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 573/1161] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:37 2022] GET /identity/v3/role_assignments?user.id=acd469f7110f46108444e99547c9c0b7&scope.project.id=9e3e6e52c95e46d78cd08b915e68395e&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:40.162603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e012d70-3ab7-4779-a269-9b441b77e1ff None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.163172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e012d70-3ab7-4779-a269-9b441b77e1ff None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.163172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e012d70-3ab7-4779-a269-9b441b77e1ff None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.164548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 589/1162] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:40.169677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.170100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.170211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.186937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:40.188690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:40.251923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 574/1163] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:40.274166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e012d70-3ab7-4779-a269-9b441b77e1ff None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:40.277277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:40.373323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e012d70-3ab7-4779-a269-9b441b77e1ff None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:40.407497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:40.408147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.408432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.408711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.474032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:40.475435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:40.477192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:40.570919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 590/1164] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:40.576909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:40.672352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e9bef9-9343-423f-bc3b-86a5e1292139 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:40.688652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:40.705121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:40.705902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.706187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.706464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.712952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:40.714385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-ddbcd12f-593a-4133-ab22-25805492dc44 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:40.728706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:40.729872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:40.731056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 575/1165] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:30:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:40.736236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:40.831530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088e5ac-2519-4399-b233-870fa54e32dc None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:40.833034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:40.865920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:40.866576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:40.866865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:40.867147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:40.867895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:40.868205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:40.869575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d067118-26d8-4795-8e9a-bba8c7d5ec11 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:40.876574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 591/1166] 10.222.0.43 () {56 vars in 1115 bytes} [Fri Jun 24 08:30:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:42.868120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1427a-1f37-4972-9636-7b6ab150c573 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:42.868646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1427a-1f37-4972-9636-7b6ab150c573 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:42.868646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1427a-1f37-4972-9636-7b6ab150c573 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:42.870041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 576/1167] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:30:42.875456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a13f-93e8-47ea-a870-a75c692915da None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:42.875874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a13f-93e8-47ea-a870-a75c692915da None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:42.876026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3925a13f-93e8-47ea-a870-a75c692915da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:42.895364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3925a13f-93e8-47ea-a870-a75c692915da None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:42.896543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3925a13f-93e8-47ea-a870-a75c692915da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:42.962021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 592/1168] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:30:42.986127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1427a-1f37-4972-9636-7b6ab150c573 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:43.085478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1427a-1f37-4972-9636-7b6ab150c573 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:43.118222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a9d69-a431-4a85-a8ed-e23b439b91c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:43.118806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a9d69-a431-4a85-a8ed-e23b439b91c9 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:43.119092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a9d69-a431-4a85-a8ed-e23b439b91c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:43.119400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a9d69-a431-4a85-a8ed-e23b439b91c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:43.120135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a9d69-a431-4a85-a8ed-e23b439b91c9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:43.121507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a9d69-a431-4a85-a8ed-e23b439b91c9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:43.136053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 577/1169] 10.222.0.43 () {60 vars in 1138 bytes} [Fri Jun 24 08:30:42 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:45.165331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9bf5-a8d8-469f-aa01-3252269a7c6a None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.165851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9bf5-a8d8-469f-aa01-3252269a7c6a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.165941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835e9bf5-a8d8-469f-aa01-3252269a7c6a None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.167319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 593/1170] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:45.173124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.173527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.173681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.190058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:45.191274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:45.220068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:45.257124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 578/1171] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:45.280290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e9bf5-a8d8-469f-aa01-3252269a7c6a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:45.372557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:45.379641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835e9bf5-a8d8-469f-aa01-3252269a7c6a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:45.413366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7fd4f-6f02-4ee2-b72f-d10c5586b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:45.414209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7fd4f-6f02-4ee2-b72f-d10c5586b99d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.414490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7fd4f-6f02-4ee2-b72f-d10c5586b99d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.414763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7fd4f-6f02-4ee2-b72f-d10c5586b99d None admin] PATH_INFO: `/v3/domains/59788c367c664625a527284dbb58a3cd` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.424975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7fd4f-6f02-4ee2-b72f-d10c5586b99d None admin] RBAC: Authorizing `identity:get_domain(domain_id=59788c367c664625a527284dbb58a3cd)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:45.426364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7fd4f-6f02-4ee2-b72f-d10c5586b99d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:45.428002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 594/1172] 10.222.0.43 () {56 vars in 1183 bytes} [Fri Jun 24 08:30:45 2022] GET /identity/v3/domains/59788c367c664625a527284dbb58a3cd => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:45.433752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:45.528574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da9458-7045-4494-94ba-619ce7ecdaba None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:45.560735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db938211-44c5-4ffb-9f83-a724e9a6bc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:45.561368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db938211-44c5-4ffb-9f83-a724e9a6bc85 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:45.561646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db938211-44c5-4ffb-9f83-a724e9a6bc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:45.561918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db938211-44c5-4ffb-9f83-a724e9a6bc85 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:45.562610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db938211-44c5-4ffb-9f83-a724e9a6bc85 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:45.564051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db938211-44c5-4ffb-9f83-a724e9a6bc85 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:45.588119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 579/1173] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:30:45 2022] POST /identity/v3/projects => generated 361 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:47.685581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c30a-65fe-4986-8831-53df0b60e165 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:47.686123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c30a-65fe-4986-8831-53df0b60e165 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:47.686123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c30a-65fe-4986-8831-53df0b60e165 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:47.687609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 595/1174] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:47.692779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:47.693203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:47.693336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:47.710256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:47.711450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:47.725455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:47.776362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 580/1175] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:47.801757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c30a-65fe-4986-8831-53df0b60e165 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:47.900670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c30a-65fe-4986-8831-53df0b60e165 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:47.932637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf098dd-d7bb-4402-beb8-ce4895717efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:47.933493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf098dd-d7bb-4402-beb8-ce4895717efa None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:47.933777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf098dd-d7bb-4402-beb8-ce4895717efa None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:47.934051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf098dd-d7bb-4402-beb8-ce4895717efa None admin] PATH_INFO: `/v3/domains/59788c367c664625a527284dbb58a3cd` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:47.935235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf098dd-d7bb-4402-beb8-ce4895717efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=59788c367c664625a527284dbb58a3cd)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:47.936733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf098dd-d7bb-4402-beb8-ce4895717efa None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:47.938788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 596/1176] 10.222.0.43 () {56 vars in 1183 bytes} [Fri Jun 24 08:30:47 2022] GET /identity/v3/domains/59788c367c664625a527284dbb58a3cd => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:47.944751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:48.038410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9e79-48b0-49a8-9132-0a5413f517fa None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:48.070785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6535d78-849a-4b1b-b584-fae6d71cb16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:48.071464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6535d78-849a-4b1b-b584-fae6d71cb16d None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:48.071745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6535d78-849a-4b1b-b584-fae6d71cb16d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:48.072020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6535d78-849a-4b1b-b584-fae6d71cb16d None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:48.072717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6535d78-849a-4b1b-b584-fae6d71cb16d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:48.074144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6535d78-849a-4b1b-b584-fae6d71cb16d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:48.093853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 581/1177] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:30:47 2022] POST /identity/v3/users => generated 311 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:30:50.143176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c109b-1ae8-4da1-ac18-67f82cbca1e0 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.143987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c109b-1ae8-4da1-ac18-67f82cbca1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.143987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416c109b-1ae8-4da1-ac18-67f82cbca1e0 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.145143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 597/1178] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:50.150507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.150898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.151068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.162605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.168559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:50.170116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:50.238131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 582/1179] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:50.261327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c109b-1ae8-4da1-ac18-67f82cbca1e0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.286663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.360698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416c109b-1ae8-4da1-ac18-67f82cbca1e0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.392096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.392875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.393158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.393486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.403873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:50.411827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:50.412918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:50.413975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 598/1180] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:50.418818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.512248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a2c56b-4368-44fd-bfb3-2e029a3aceb8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.544653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.545303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.545585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.545860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.547173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:50.547505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.548868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.550966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.557145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 583/1181] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:50.562632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.642749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.658979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79541fc5-3338-42b5-9fe1-465e4fb16087 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.690868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.691749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.692030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.692306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] PATH_INFO: `/v3/users/d1543962e608428a98d3d814dbd86181` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.705177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] RBAC: Authorizing `identity:get_user(user_id=d1543962e608428a98d3d814dbd86181)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.706593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:50.708214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 599/1182] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/users/d1543962e608428a98d3d814dbd86181 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:50.713804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:50.809469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6db9cd-aa0b-4326-8fa0-1e0f86e04f8f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:50.841526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:50.842156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:50.842439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:50.842716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:50.905435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:50.909344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:50.910987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:51.003832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 584/1183] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:51.009744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:51.105799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8ce6d7-580c-46f7-8aa9-db18025c7f65 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:51.137904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e904e5-7b8c-4a7d-9cbc-23413f7c8767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:51.138736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e904e5-7b8c-4a7d-9cbc-23413f7c8767 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:51.139016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e904e5-7b8c-4a7d-9cbc-23413f7c8767 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:51.139311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e904e5-7b8c-4a7d-9cbc-23413f7c8767 None admin] PATH_INFO: `/v3/projects/83f4619ef6924739be276024d3fa04ba` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:51.141023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e904e5-7b8c-4a7d-9cbc-23413f7c8767 None admin] RBAC: Authorizing `identity:get_project(project_id=83f4619ef6924739be276024d3fa04ba)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:51.142434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e904e5-7b8c-4a7d-9cbc-23413f7c8767 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:51.144096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 600/1184] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:51 2022] GET /identity/v3/projects/83f4619ef6924739be276024d3fa04ba => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:51.150191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:51.242021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:51.245750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abe0d6-24a6-4faf-a27f-9abcd2e80b58 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:51.280484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:51.281140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:51.281425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:51.281704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:51.282528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=83f4619ef6924739be276024d3fa04ba, user.id=d1543962e608428a98d3d814dbd86181) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:51.282843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:51.284312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18bafef-d866-4207-9a8f-2fbc3e2505cd None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:51.289592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 585/1185] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:51 2022] GET /identity/v3/role_assignments?user.id=d1543962e608428a98d3d814dbd86181&scope.project.id=83f4619ef6924739be276024d3fa04ba&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:30:53.270383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be3031-7880-48b0-95c2-c12c4eade9d0 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.270935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be3031-7880-48b0-95c2-c12c4eade9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.270935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6be3031-7880-48b0-95c2-c12c4eade9d0 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.272419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 601/1186] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:53.277884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.278340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.278475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.295304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:53.296483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:53.360937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 586/1187] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:53.383460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be3031-7880-48b0-95c2-c12c4eade9d0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:53.395475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:53.482125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6be3031-7880-48b0-95c2-c12c4eade9d0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:53.514343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:53.515117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.515426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.515705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.525694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:53.533636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:53.534796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:30:53.535757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 602/1188] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:53.540731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:53.629871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:53.638742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65a0aa-c538-4d1e-b691-578300d83d79 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:53.671265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:53.672424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.672732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.673009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.673749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:53.674055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:53.675436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:53.683178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 587/1189] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:53.689283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:53.755823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:53.786283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f36243-9d04-4533-b158-ddd05c76668e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:53.818925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:53.819806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.820091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.820368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] PATH_INFO: `/v3/users/d1543962e608428a98d3d814dbd86181` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.821537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] RBAC: Authorizing `identity:get_user(user_id=d1543962e608428a98d3d814dbd86181)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:53.822954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:53.824574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 603/1190] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:53 2022] GET /identity/v3/users/d1543962e608428a98d3d814dbd86181 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:53.830057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:53.929008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089c2bc0-473f-44f1-ab9a-797d201b876b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:53.961246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6582ae3-6920-48df-a7cf-197e82abeeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:53.962097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6582ae3-6920-48df-a7cf-197e82abeeed None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:53.962379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6582ae3-6920-48df-a7cf-197e82abeeed None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:53.962653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6582ae3-6920-48df-a7cf-197e82abeeed None admin] PATH_INFO: `/v3/projects/83f4619ef6924739be276024d3fa04ba` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:53.963830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6582ae3-6920-48df-a7cf-197e82abeeed None admin] RBAC: Authorizing `identity:get_project(project_id=83f4619ef6924739be276024d3fa04ba)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:53.965334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6582ae3-6920-48df-a7cf-197e82abeeed None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:53.967025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 588/1191] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:53 2022] GET /identity/v3/projects/83f4619ef6924739be276024d3fa04ba => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:53.973291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:54.068958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c3d6c-d342-4a77-93ce-5cc59ba8f059 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:54.101128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40743b3c-37c3-4839-9f8a-d3283e39efaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:54.102002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743b3c-37c3-4839-9f8a-d3283e39efaf None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:54.102282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743b3c-37c3-4839-9f8a-d3283e39efaf None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:54.102557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40743b3c-37c3-4839-9f8a-d3283e39efaf None admin] PATH_INFO: `/v3/projects/83f4619ef6924739be276024d3fa04ba/users/d1543962e608428a98d3d814dbd86181/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:54.104510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40743b3c-37c3-4839-9f8a-d3283e39efaf None admin] RBAC: Authorizing `identity:create_grant(project_id=83f4619ef6924739be276024d3fa04ba, user_id=d1543962e608428a98d3d814dbd86181, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:54.105927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40743b3c-37c3-4839-9f8a-d3283e39efaf None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:54.115276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:54.121162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 604/1192] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:30:53 2022] PUT /identity/v3/projects/83f4619ef6924739be276024d3fa04ba/users/d1543962e608428a98d3d814dbd86181/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:30:56.107471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e57dd-a611-4553-a29e-215f889a55b2 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.108025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e57dd-a611-4553-a29e-215f889a55b2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.108025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e57dd-a611-4553-a29e-215f889a55b2 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.109867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 589/1193] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:30:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:30:56.115027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.115537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.115758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.132731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:30:56.133915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:30:56.198811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 605/1194] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:30:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:30:56.222650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e57dd-a611-4553-a29e-215f889a55b2 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.246940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:56.322144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e57dd-a611-4553-a29e-215f889a55b2 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.353919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.354689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.354970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.355262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.366020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:30:56.373470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:30:56.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:30:56.375877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 590/1195] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:30:56.380536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.476104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed51757-03db-404b-ae72-78f7834c295f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.501726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:56.509754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.510416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.510701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.510980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.511758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:56.512072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.513438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.521294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 606/1196] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:56.527457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.606336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:56.624257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1688b70b-954b-484b-91b1-7f9315bc5cab None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.657466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.658314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.658598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.658873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] PATH_INFO: `/v3/users/d1543962e608428a98d3d814dbd86181` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.660068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] RBAC: Authorizing `identity:get_user(user_id=d1543962e608428a98d3d814dbd86181)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.661485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.663073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 591/1197] 10.222.0.43 () {56 vars in 1177 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/users/d1543962e608428a98d3d814dbd86181 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:56.668684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:56.763449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff2104-3ab8-42c0-bc15-f8ac635e6859 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:56.796242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:56.796882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:56.797165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:56.797440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:56.852373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:56.862484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:56.863938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:56.959032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 607/1198] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:30:56.964755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:57.066675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cee1e-efaa-40e1-addb-65fe10f74f24 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:57.099178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25080bd-a62c-4ee3-bac8-75987f33a85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:57.100047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25080bd-a62c-4ee3-bac8-75987f33a85b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:57.100327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25080bd-a62c-4ee3-bac8-75987f33a85b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:57.100601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25080bd-a62c-4ee3-bac8-75987f33a85b None admin] PATH_INFO: `/v3/projects/83f4619ef6924739be276024d3fa04ba` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:57.101779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25080bd-a62c-4ee3-bac8-75987f33a85b None admin] RBAC: Authorizing `identity:get_project(project_id=83f4619ef6924739be276024d3fa04ba)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:57.103204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25080bd-a62c-4ee3-bac8-75987f33a85b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:57.104945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 592/1199] 10.222.0.43 () {56 vars in 1186 bytes} [Fri Jun 24 08:30:56 2022] GET /identity/v3/projects/83f4619ef6924739be276024d3fa04ba => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:30:57.110864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:30:57.198678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:30:57.208771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c64b9d-67c9-4938-a4c4-edffc3b140c5 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:30:57.241085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:30:57.241673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:30:57.241953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:30:57.242225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:30:57.243048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=83f4619ef6924739be276024d3fa04ba, user.id=d1543962e608428a98d3d814dbd86181) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:30:57.243385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:30:57.244804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800f3e4d-e61a-4885-bf7f-a9f4ac079fc6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:30:57.250931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 608/1200] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:30:57 2022] GET /identity/v3/role_assignments?user.id=d1543962e608428a98d3d814dbd86181&scope.project.id=83f4619ef6924739be276024d3fa04ba&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:07.095142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:07.247847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=23 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:52.455489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd606ab-3ce2-4441-9f25-8b9fa7ad47ab None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:52.456059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd606ab-3ce2-4441-9f25-8b9fa7ad47ab None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:52.456059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd606ab-3ce2-4441-9f25-8b9fa7ad47ab None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:52.457333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 593/1201] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:31:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:52.462606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:52.463137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:52.463137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:52.480816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:52.482010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:52.576325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 609/1202] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:52.604361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd606ab-3ce2-4441-9f25-8b9fa7ad47ab None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:52.727739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd606ab-3ce2-4441-9f25-8b9fa7ad47ab None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:52.771522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:52.772276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:52.772559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:52.772877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:52.802693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:52.804322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:52.824778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:52.825892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:52.827074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:52.827074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:52.827074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:52.827074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:31:52.827074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 594/1203] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:31:52.833089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:52.970189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a7ddc-6032-4d3c-b37b-11004a05c23d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:53.011875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:53.021071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:53.021757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:53.022039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:53.022325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:53.023053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:53.023422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:53.024972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd1c51-bf4b-4ba8-a227-c92fa1c8f948 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:53.042553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 610/1204] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:53.043877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:53.111365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:53.172139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b229d275-25e3-450a-8cf2-1c64cecdf36f None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:53.216616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6603e6b1-a1be-4229-aad8-c3e3860343a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:53.217348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603e6b1-a1be-4229-aad8-c3e3860343a9 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:53.217664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603e6b1-a1be-4229-aad8-c3e3860343a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:53.217970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6603e6b1-a1be-4229-aad8-c3e3860343a9 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:53.218796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603e6b1-a1be-4229-aad8-c3e3860343a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:53.221112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6603e6b1-a1be-4229-aad8-c3e3860343a9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:53.249187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 595/1205] 10.222.0.43 () {60 vars in 1132 bytes} [Fri Jun 24 08:31:53 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:31:55.568396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbe292-f360-4953-9aab-aacd7011cfff None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:55.568396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbe292-f360-4953-9aab-aacd7011cfff None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:55.568396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbe292-f360-4953-9aab-aacd7011cfff None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:55.568396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 611/1206] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:31:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:31:55.573591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:55.574095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:55.574363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:55.594734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:31:55.596064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:31:55.681928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 596/1207] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:31:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:31:55.709849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbe292-f360-4953-9aab-aacd7011cfff None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:55.839443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbe292-f360-4953-9aab-aacd7011cfff None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:55.877893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:55.878900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:55.879318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:55.879667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:55.893921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:31:55.901889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:55.907389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:31:55.908543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 612/1208] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:31:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:55.910927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:55.963978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:56.028480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb899b1-2de3-4530-aa2d-62b251cbf9b3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:56.073161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:56.073884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:56.074204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:56.074487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:56.075263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:56.075573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:56.077026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:56.091753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 597/1209] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:31:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:56.096314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:56.158823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:56.217679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f73b2-19cf-4588-ba90-297600fae137 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:56.262042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:56.263025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:56.263410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:56.263791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:56.276765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:56.278535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:56.298190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:56.299306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:56.300506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:56.300506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:56.300506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:56.300506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:31:56.304266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 613/1210] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:56.305252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:56.412609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:56.416354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7432b03-667f-4696-b2a8-f3b3aa18ef1a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:56.474781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:56.475641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:56.476021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:56.476348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:56.477103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:56.477415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:56.478980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:56.496052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 598/1211] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:31:56.501957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:56.622616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba8ba8-fa9d-4b33-bca2-c65f8cacf5b7 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:56.624473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:56.667109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:56.667960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:56.668264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:56.668528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:56.677964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:56.679602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:31:56.697488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:56.700928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:31:56.701947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:31:56.705435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 614/1212] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:56.705570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:56.821304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b790f8c-88ea-41ed-a888-b07200b30b24 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:56.864638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:56.865327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:56.865622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:56.865896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:56.866844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:56.867153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:56.868794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:56.892137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 599/1213] 10.222.0.43 () {56 vars in 1143 bytes} [Fri Jun 24 08:31:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:56.897881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:57.021643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8cdf4-a169-4f4f-bf3e-3eaa2fe30490 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:57.060556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:57.061368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:57.061661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:57.061945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:57.104136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:57.153853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:57.155563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:57.265515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 615/1214] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:31:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:31:57.272161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:57.292117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:57.377097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df149a4-0166-47a4-b887-38e1940844a8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:57.412053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:57.412850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:57.413169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:57.413403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:57.426652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:57.428203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:31:57.449670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:57.450868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:57.452055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:31:57.452055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:31:57.452055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:31:57.452055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:31:57.453595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 600/1215] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:31:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:31:57.459338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:57.530949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:57.571261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc4707-bbbb-487d-92e5-8e9b5f50e379 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:57.606583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:57.607328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:57.607559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:57.607847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:57.608559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:57.608867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:57.610760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:57.623253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 616/1216] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:31:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:57.629667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:57.717148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:57.751578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd0b43c-d639-4499-8ef9-664369acdf20 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:57.787183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:57.788128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:57.788516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:57.788884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:57.796134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:57.798263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:31:57.812660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:31:57.813856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:31:57.815012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:31:57.816730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 601/1217] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:31:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:31:57.822883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:57.933637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1441dccf-5822-4c44-a970-16deb2312a5c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:57.962935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:57.977318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:57.978096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:57.978357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:57.978593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:57.979633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:57.979980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:57.981404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a54cf79-4c88-4112-bc4a-f1c5f8f9197e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:57.994660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 617/1218] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:31:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:31:58.000559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:31:58.115432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856d310-e999-469d-b208-2a83ffd0b55a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:31:58.145489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:31:58.152347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:31:58.153246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:31:58.153571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:31:58.153842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:31:58.154771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=a39a333f4c42420cb46189d32aafc6fb) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:31:58.155086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:31:58.157176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a11ee0d-1c6a-4152-bcf9-bcf3c3f20465 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:31:58.171256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 602/1219] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:31:57 2022] GET /identity/v3/role_assignments?user.id=a39a333f4c42420cb46189d32aafc6fb&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:32:00.305274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c28b9-d35b-406c-b44d-5564490e30b3 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:00.305888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c28b9-d35b-406c-b44d-5564490e30b3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:00.305888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6c28b9-d35b-406c-b44d-5564490e30b3 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:00.307400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 618/1220] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:00.313318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:00.313745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:00.313937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:00.332133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:00.333358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:00.405340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 603/1221] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:00.430295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c28b9-d35b-406c-b44d-5564490e30b3 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:00.542245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6c28b9-d35b-406c-b44d-5564490e30b3 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:00.578904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:00.579789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:00.580128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:00.580454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:00.591738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:32:00.600676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:00.601829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:00.602992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 619/1222] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:32:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:00.608441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:00.713024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3fb50-d51a-4950-936b-724413186a69 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:00.737093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:00.748026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:00.748690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:00.748970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:00.749244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:00.749990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] RBAC: Adding query filter params (name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:00.750821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:00.752963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:00.761526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 604/1223] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:32:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:00.768823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:00.817070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:00.876202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c33c3-a205-4e08-833d-e56367121fab None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:00.915194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:00.916017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:00.916301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:00.916577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:00.927320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:00.928752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:00.947498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:00.948588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:00.949805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:00.949805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:00.949805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:00.949805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:00.949805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 620/1224] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:00.955237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:01.070035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee707731-5712-4fb9-97c5-1e19e39e3274 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:01.129476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:01.130370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:01.130848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:01.131313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:01.132484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:01.132778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:01.134427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:01.154728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 605/1225] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:01.160397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:01.287683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:01.295705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a0ec2-4823-43c2-afed-d9c6479f37cb None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:01.335676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:01.336492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:01.336675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:01.337058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:01.346310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:01.348076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:32:01.365149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:01.367489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:01.368790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 621/1226] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:32:01.372729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:01.430758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:01.488407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c4cc00-e470-491a-83ff-e4f80473d993 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:01.525783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:01.526448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:01.526724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:01.526996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:01.527852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:01.528157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:01.529575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:01.548607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 606/1227] 10.222.0.43 () {56 vars in 1143 bytes} [Fri Jun 24 08:32:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:32:01.554562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:01.666838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e09816-1aea-47ba-a7b3-e7836ad60b37 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:01.701689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:01.702667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:01.702957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:01.703283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:01.714980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:01.716590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:01.736003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:01.737174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:01.738114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:01.738114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:01.738114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:01.738114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:01.738114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 622/1228] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:32:01.743614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:01.857306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1073b-c9c2-4c91-98fc-67d457389be7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:01.859647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:01.899249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:01.900143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:01.900829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:01.901131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:01.901952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:01.902335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:01.904010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:01.918332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 607/1229] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:01.923908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:01.933642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:02.033931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef37f4-3fa6-4b14-a8e3-f8df54a42fd9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:02.069732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:02.070533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:02.070862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:02.071094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:02.078131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:02.080153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:32:02.096664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:02.097924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:02.098951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 623/1230] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:32:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:02.104582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:02.226557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9f55b-63f6-43a3-bda3-fc28ba1c5e4d None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:02.264531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:02.265195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:02.265482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:02.265806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:02.267090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:02.267497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:02.269058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213db4a4-b497-496e-8843-925dbd0a3fd0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:02.282594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 608/1231] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:32:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:02.288646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:02.360609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:02.396897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b3c04-c6e9-4430-9fa8-ad3bbbf0c42c None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:02.434029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4dc121-2af5-4090-a209-30fbeb77218d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:02.434847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dc121-2af5-4090-a209-30fbeb77218d None admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:02.435129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dc121-2af5-4090-a209-30fbeb77218d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:02.435428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4dc121-2af5-4090-a209-30fbeb77218d None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/a39a333f4c42420cb46189d32aafc6fb/roles/c268071abd644ea4930f864f32d4054a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:02.451157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4dc121-2af5-4090-a209-30fbeb77218d None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=a39a333f4c42420cb46189d32aafc6fb, role_id=c268071abd644ea4930f864f32d4054a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:02.452825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4dc121-2af5-4090-a209-30fbeb77218d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:02.467023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 624/1232] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:32:02 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/a39a333f4c42420cb46189d32aafc6fb/roles/c268071abd644ea4930f864f32d4054a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:32:04.619192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533188c-cb1b-4948-92d5-9769d6dd4214 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:04.619771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533188c-cb1b-4948-92d5-9769d6dd4214 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:04.619771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e533188c-cb1b-4948-92d5-9769d6dd4214 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:04.621171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 609/1233] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:04.626995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:04.627539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:04.627631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:04.645918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:04.647271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:04.719945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 625/1234] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:04.745686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533188c-cb1b-4948-92d5-9769d6dd4214 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:04.771818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:04.868346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e533188c-cb1b-4948-92d5-9769d6dd4214 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:04.909265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:04.909985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:04.910409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:04.910549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:04.922587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:32:04.932358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:04.933389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:04.934262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 610/1235] 10.222.0.43 () {56 vars in 1102 bytes} [Fri Jun 24 08:32:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:04.940233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:04.991803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:05.049890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b149743-7bbf-4be8-8e60-c09e908e6068 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:05.085974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:05.086685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:05.087010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:05.087354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:05.088352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] RBAC: Adding query filter params (name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:05.088716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:05.090078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:05.099950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 626/1236] 10.222.0.43 () {56 vars in 1103 bytes} [Fri Jun 24 08:32:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:32:05.105474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:05.204727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:05.222455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0e666-56ab-48b1-8c2c-c0ce6f850453 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:05.259160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:05.259986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:05.260270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:05.260550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:05.271462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:05.272912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:05.290902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:05.292091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:05.293140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:05.293140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:05.293140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:05.293140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:05.293140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 611/1237] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:05.298565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:05.417403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ee3ed-a6b2-4929-8fe9-68709e16fd8e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:05.431862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:05.462385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:05.463049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:05.463364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:05.463641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:05.464402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:05.464710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:05.466069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:05.479656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 627/1238] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:32:05.486758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:05.598396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc42d69-1883-48a1-8799-750211689394 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:05.633831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:05.634663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:05.634950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:05.635262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:05.638491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:05.643818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:05.645689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:32:05.661167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:05.662382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:05.663887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 612/1239] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:05.668415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:05.784690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7892c017-cea8-44cc-b723-4ee5d9dc727e None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:05.823806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:05.824485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:05.824790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:05.825113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:05.825979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:05.826293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:05.827756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:05.846553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 628/1240] 10.222.0.43 () {56 vars in 1143 bytes} [Fri Jun 24 08:32:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:05.852492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:05.980881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa584b92-26a9-4039-a943-d0b75b180e6b None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:06.021064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:06.021713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:06.021995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:06.022278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:06.090911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:06.100649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:06.102433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:06.210723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 613/1241] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:32:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:32:06.216615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:06.243788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:06.322994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80dadd02-9924-46a0-b38b-8f785833a472 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:06.358229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:06.359174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:06.360093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:06.360421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:06.370700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:06.372218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:06.390450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:06.391635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:06.392716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:06.392716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:06.392716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:06.392716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:06.392716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 629/1242] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:06.397651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:06.507013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:06.519440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f90d13-94aa-460d-a6be-78acf67eadf2 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:06.569005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:06.569701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:06.569989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:06.570266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:06.571057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:06.571409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:06.572875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:06.585910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 614/1243] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:32:06.591858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:06.680225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:06.705537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3a4b79-40ea-411c-8d06-daf45a524b30 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:06.745141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:06.746092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:06.746478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:06.746784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:06.757740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:06.759290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:32:06.774373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:06.776023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:06.777521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 630/1244] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:32:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:06.782486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:06.901812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbec7e91-ce4d-48d2-95f8-44dd699c64bb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:06.943796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:06.944453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:06.944774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:06.945074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:06.945950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:06.946243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:06.947878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21ea8b-da0d-417c-aa22-fe25050619ad None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:06.962344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 615/1245] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:32:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:06.969822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:07.080796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5137575a-2eed-40c7-8837-672088dcc5b2 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:07.116565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:07.117217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:07.117495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:07.117765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:07.118590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] RBAC: Adding query filter params (role.id=c268071abd644ea4930f864f32d4054a, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=a39a333f4c42420cb46189d32aafc6fb) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:07.118920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:07.120351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3a27a-3a36-400b-848c-c6be995fbc53 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:07.126542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 631/1246] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:32:06 2022] GET /identity/v3/role_assignments?user.id=a39a333f4c42420cb46189d32aafc6fb&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=c268071abd644ea4930f864f32d4054a => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:09.346937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac723d-6997-4008-9a33-1b71c7ef67b7 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:09.347567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac723d-6997-4008-9a33-1b71c7ef67b7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:09.347679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ac723d-6997-4008-9a33-1b71c7ef67b7 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:09.349027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 616/1247] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:09.354582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:09.355160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:09.355160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:09.361066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:09.375269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:09.376648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:09.455941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 632/1248] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 24 08:32:09.484350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac723d-6997-4008-9a33-1b71c7ef67b7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:09.505668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:09.606193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ac723d-6997-4008-9a33-1b71c7ef67b7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:09.647455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:09.648280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:09.648590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:09.648930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:09.662790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:32:09.671247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:09.672410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:09.673273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 617/1249] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:32:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:32:09.679201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:09.788333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd63b81-7d1c-46c6-8df9-dd82d925538a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:09.817957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:09.823969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:09.824730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:09.825072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:09.825352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:09.826140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:09.826560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:09.828041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:09.836961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 633/1250] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:32:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:32:09.843668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:09.942016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:09.969432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb784896-b716-4a44-a844-bfba770195a1 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:10.006201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:10.007775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:10.008482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:10.008818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:10.020476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:10.021968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:10.038744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:10.039838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:10.040774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:10.040774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:10.040774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:10.040774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:10.040774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 618/1251] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:10.045665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:10.157358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5346e-21ba-4889-8e0d-e816621ef506 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:10.205309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:10.205979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:10.206264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:10.206543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:10.207292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:10.207606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:10.208962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:10.222113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 634/1252] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:10.228111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:10.343466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f098bb4-5a96-4ea8-b9da-53007b0aed58 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:10.376001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:10.385547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:10.386355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:10.386637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:10.386914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:10.395390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:10.396833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:32:10.413406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:10.414905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:10.416076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:10.420826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 619/1253] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:10.421886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:10.480460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:10.527384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be19ec0-5a9c-437f-9146-5455e88113d4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:10.564405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:10.565090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:10.565378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:10.565698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:10.566812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:10.567150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:10.568800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:10.589846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 635/1254] 10.222.0.43 () {56 vars in 1143 bytes} [Fri Jun 24 08:32:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:10.596031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:10.710875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119f7b0-fd70-4c25-b65f-9a33e9d10938 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:10.747658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:10.748308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:10.748589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:10.748866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:10.809330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:10.823129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:10.824609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:10.941116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 620/1255] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:32:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 24 08:32:10.948242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:11.064718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7acaa35-2b34-4a44-aad9-1c190a70059d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:11.078971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:11.102649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:11.103582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:11.103884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:11.104192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:11.115420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:11.116857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:11.135496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:11.136569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:11.137682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:11.137682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:11.137682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:11.137682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:11.137682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 636/1256] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:11.142527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:11.248840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:11.262210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5df4fc8-2876-45ab-adb6-d70f2b0633bb None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:11.299204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:11.299872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:11.300158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:11.300435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:11.301164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:11.301463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:11.302833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:11.315889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 621/1257] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:11.321614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:11.427770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4eb3e9-0120-4b24-8ff0-8a454a266162 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:11.468503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:11.469825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:11.470107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:11.470379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:11.477912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:11.479410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:32:11.495135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:11.496336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:11.497465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 637/1258] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:32:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 24 08:32:11.502726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:11.607980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f47fa-b2f4-4353-afd1-6915132e6517 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:11.642356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:11.643037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:11.643351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:11.643631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:11.644438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:11.644750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:11.646238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a5a0c-1784-4a5e-b670-e70680a2d121 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:11.660285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 622/1259] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:32:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:11.666695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:11.693239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:11.771863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e6b1-46be-4204-8249-c867330e0568 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:11.805610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:11.806268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:11.806555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:11.806845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:11.807704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=a39a333f4c42420cb46189d32aafc6fb) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:11.808020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:11.809466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058c280-bc40-4c41-92a4-a2bf08568070 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:11.815321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 638/1260] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:32:11 2022] GET /identity/v3/role_assignments?user.id=a39a333f4c42420cb46189d32aafc6fb&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:13.832356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d63b23-482e-43c6-a813-32c0a4bd12d6 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:13.832953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d63b23-482e-43c6-a813-32c0a4bd12d6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:13.832953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d63b23-482e-43c6-a813-32c0a4bd12d6 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:13.834281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 623/1261] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:13.839343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:13.839753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:13.839910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:13.856270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:13.857436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:13.932020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 639/1262] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:13.956326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d63b23-482e-43c6-a813-32c0a4bd12d6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:13.967046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:14.055755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d63b23-482e-43c6-a813-32c0a4bd12d6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:14.088737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:14.089461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:14.089744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:14.090023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:14.100597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:32:14.108120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:14.109233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:14.110949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 624/1263] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:32:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:14.115730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:14.134127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:14.214375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ee834-4d08-454b-be12-3971eb36ce31 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:14.246464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:14.247125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:14.247438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:14.247718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:14.248465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:14.248774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:14.250645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:14.258447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 640/1264] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:32:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:14.264102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:14.328781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:14.360577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39038dff-b132-4a91-a088-6c4265ca94b0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:14.392929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:14.393705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:14.393983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:14.394254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:14.404713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:14.406165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:14.422615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:14.423705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:14.424971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:14.424971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:14.424971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:14.424971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:14.424971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 625/1265] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:14.429444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:14.509565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:14.529124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246eec9b-bed5-42c5-9179-17321a9acc87 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:14.561810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:14.562517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:14.562805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:14.563086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:14.563838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:14.564146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:14.565508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:14.576908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 641/1266] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:14.582970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:14.680187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5755888-ea7b-4469-b01b-4e898264d8c7 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:14.698719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:14.712983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:14.713770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:14.714053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:14.714329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:14.722162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:14.723642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:32:14.737557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:14.738627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:14.744272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 626/1267] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:14.745008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:14.838667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04b1cc-370a-4aa1-a789-f18362a29ae4 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:14.870922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:14.871601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:14.871881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:14.872156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:14.872975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:14.873286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:14.874957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:14.885103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:14.896436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 642/1268] 10.222.0.43 () {56 vars in 1143 bytes} [Fri Jun 24 08:32:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:14.903114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:15.003355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbac8d81-524c-407e-a497-ca2c840a4957 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:15.035985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:15.036777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:15.037062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:15.037340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:15.047470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:15.048910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:15.066035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:15.067093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:15.068096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:15.068096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:15.068096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:15.068096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:15.068096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 627/1269] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:15.072848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:15.169610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd28f45-6612-465b-b118-4146d19c06b6 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:15.204174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:15.204839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:15.205123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:15.205400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:15.206118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:15.206424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:15.208157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:15.219053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 643/1270] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:15.224572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:15.255199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:15.321454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4e7b8a-765b-4e6d-b86d-c4a3fbb16d94 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:15.352948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:15.353722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:15.354005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:15.354279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:15.361236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:15.362675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:32:15.376945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:15.377986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:15.379250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 628/1271] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:32:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:15.383715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:15.440642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:15.481515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b600e9a8-9100-4b1e-a347-9979e4a79b58 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:15.513405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:15.514001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:15.514287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:15.514571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:15.515386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:15.516223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:15.517650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a1748-2fe1-43b3-9654-8de62bf3e48b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:15.529473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 644/1272] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:32:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:15.535907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:15.622843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:15.631731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c501d7d-2682-4a97-a2ad-89e067a2767a None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:15.665380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f169002-24dd-4790-9aa2-a44e01871f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:15.666244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f169002-24dd-4790-9aa2-a44e01871f1f None admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:15.666532 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f169002-24dd-4790-9aa2-a44e01871f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:15.666808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f169002-24dd-4790-9aa2-a44e01871f1f None admin] PATH_INFO: `/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/a39a333f4c42420cb46189d32aafc6fb/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:15.668750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f169002-24dd-4790-9aa2-a44e01871f1f None admin] RBAC: Authorizing `identity:create_grant(project_id=95ac8a1e51604baa804e78a0d3b026ce, user_id=a39a333f4c42420cb46189d32aafc6fb, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:15.670180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f169002-24dd-4790-9aa2-a44e01871f1f None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:15.682298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 629/1273] 10.222.0.43 () {58 vars in 1439 bytes} [Fri Jun 24 08:32:15 2022] PUT /identity/v3/projects/95ac8a1e51604baa804e78a0d3b026ce/users/a39a333f4c42420cb46189d32aafc6fb/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:32:17.672074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77ce0-8377-450a-a827-737717f9b9d6 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:17.672609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77ce0-8377-450a-a827-737717f9b9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:17.672609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a77ce0-8377-450a-a827-737717f9b9d6 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:17.673980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 645/1274] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:17.678982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:17.679434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:17.679570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:17.695907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:17.697077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:17.761093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 630/1275] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:17.785702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a77ce0-8377-450a-a827-737717f9b9d6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:17.883504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a77ce0-8377-450a-a827-737717f9b9d6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:17.904397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:17.921393 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823769f0-9417-447a-951a-ea315567756d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:17.922249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823769f0-9417-447a-951a-ea315567756d None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:17.922477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823769f0-9417-447a-951a-ea315567756d None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:17.922748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823769f0-9417-447a-951a-ea315567756d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:17.932421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823769f0-9417-447a-951a-ea315567756d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-823769f0-9417-447a-951a-ea315567756d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 24 08:32:17.941495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:17.942632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:17.943751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 646/1276] 10.222.0.43 () {56 vars in 1096 bytes} [Fri Jun 24 08:32:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:17.948753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:18.026697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:18.045228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8006b38-a388-4b15-a8d6-c219bbbf90de None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:18.077485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:18.078737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:18.079017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:18.079326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:18.080075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:18.080379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:18.081736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:18.089449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 631/1277] 10.222.0.43 () {56 vars in 1099 bytes} [Fri Jun 24 08:32:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:18.094898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823769f0-9417-447a-951a-ea315567756d None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:18.189677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823769f0-9417-447a-951a-ea315567756d None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:18.221536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:18.222315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:18.222594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:18.222865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:18.232858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:18.234311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:18.251033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:18.252132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:18.253079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:18.253079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:18.253079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:18.253079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:18.253079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 647/1278] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:18.257835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:18.355168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841d6ab-e15c-48cf-a554-7920ced76706 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:18.388032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:18.388677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:18.388955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:18.389228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:18.389934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:18.390237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:18.391657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:18.403089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 632/1279] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:18.408459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:18.439098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:18.506129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8982bc-3938-4a97-892e-5bd6d6c1bc41 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:18.538805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:18.539613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:18.539898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:18.540171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:18.547935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:18.549350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 24 08:32:18.564161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:18.565285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:18.566506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 648/1280] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:18.570991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:18.574748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:18.666000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc8e488-a65b-4198-ab80-5c8177a8fba8 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:18.698154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:18.698725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:18.699007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:18.699306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:18.700124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:18.700428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:18.701838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:18.718193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 633/1281] 10.222.0.43 () {56 vars in 1143 bytes} [Fri Jun 24 08:32:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:18.724125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:18.811893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:18.821434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1c2147-4440-4224-832a-2f476565ea87 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:18.853626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:18.854273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:18.854561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:18.854840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:18.927465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:18.928895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:19.013423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:19.024075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 649/1282] 10.222.0.43 () {58 vars in 1282 bytes} [Fri Jun 24 08:32:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 24 08:32:19.030127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:19.089395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:19.126672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60f6ce1-6649-4f6d-b709-d9fb05643c7c None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:19.159873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:19.160654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:19.160941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:19.161220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:19.172803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:19.174243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 24 08:32:19.191366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:19.192524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:19.193797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 24 08:32:19.193797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 24 08:32:19.193797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 24 08:32:19.193797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:19.193797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 634/1283] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:32:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:19.198074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:19.294408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d0180-149a-462e-a045-26a79e2289d9 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:19.326578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:19.327252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:19.327541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:19.327814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:19.329029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:19.329332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:19.330676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:19.341474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 650/1284] 10.222.0.43 () {56 vars in 1109 bytes} [Fri Jun 24 08:32:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:19.347591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:19.443592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47791e2-7f9a-4ea1-a89f-430055c533bf None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:19.457462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:19.482919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:19.483730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:19.484019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:19.484300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:19.490929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:19.492427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 24 08:32:19.506283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:19.507333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: ERROR keystone.server.flask.application Jun 24 08:32:19.508444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 635/1285] 10.222.0.43 () {56 vars in 1111 bytes} [Fri Jun 24 08:32:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:19.513320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:19.565377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:19.614414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e5a1f-e47f-4468-8d4f-63301b9a804b None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:19.647671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:19.648326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:19.648612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:19.648889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:19.649676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:19.649989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:19.651706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dbff6f-7ced-4330-88de-fbaf49ca822e None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:19.663293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 651/1286] 10.222.0.43 () {56 vars in 1148 bytes} [Fri Jun 24 08:32:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:19.669267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:19.767147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e44d4-1edf-4d03-a190-bec755b13205 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:19.798883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:19.799537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:19.799823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:19.800094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:19.800909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] RBAC: Adding query filter params (role.id=b382dafaa79d4ff1afb67e7c0d7097a8, scope.project.id=95ac8a1e51604baa804e78a0d3b026ce, user.id=a39a333f4c42420cb46189d32aafc6fb) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:19.801213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:19.802622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2038238c-0390-40a4-9cec-c5e5d90aa514 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:19.808235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 636/1287] 10.222.0.43 () {56 vars in 1374 bytes} [Fri Jun 24 08:32:19 2022] GET /identity/v3/role_assignments?user.id=a39a333f4c42420cb46189d32aafc6fb&scope.project.id=95ac8a1e51604baa804e78a0d3b026ce&role.id=b382dafaa79d4ff1afb67e7c0d7097a8 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:21.851741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521c9d4-e558-4da7-bdd0-979dd5d30dd3 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:21.852278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521c9d4-e558-4da7-bdd0-979dd5d30dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:21.852278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4521c9d4-e558-4da7-bdd0-979dd5d30dd3 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:21.853665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 652/1288] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:21.858585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:21.859025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:21.859163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:21.875928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:21.877229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:21.910205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:21.949905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 637/1289] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:21.973004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521c9d4-e558-4da7-bdd0-979dd5d30dd3 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:22.072081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4521c9d4-e558-4da7-bdd0-979dd5d30dd3 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:22.082286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:22.105113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:22.105841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:22.106127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:22.106407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:22.107053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:22.108435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:22.120373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:22.121581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:22.122652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 653/1290] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:32:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:22.128032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:22.221763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ad7548-6852-459f-9e55-f079357ab8cb None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:22.254304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:22.254952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:22.255251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:22.255528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:22.256225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:22.256527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:22.257887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d91d92-b791-4e51-b548-74a97b9de2f9 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:22.262588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 638/1291] 10.222.0.43 () {56 vars in 1116 bytes} [Fri Jun 24 08:32:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:22.268248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:22.368329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2674508d-470c-4cf1-88e2-da412223e147 None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:22.400427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:22.401340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:22.401624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:22.401900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] PATH_INFO: `/v3/services` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:22.402592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] RBAC: Adding query filter params (type=placement) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:22.402897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:22.404305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9f661-18c9-419e-9f9a-8d64bfe87abb None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:22.409136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 654/1292] 10.222.0.43 () {56 vars in 1116 bytes} [Fri Jun 24 08:32:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:24.404008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74e76d-a40c-4584-a87a-ab39a750434d None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.404529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74e76d-a40c-4584-a87a-ab39a750434d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.404529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b74e76d-a40c-4584-a87a-ab39a750434d None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.405957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 639/1293] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:24.411465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1674-cca5-4439-a2af-dc457df00eb6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.411892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1674-cca5-4439-a2af-dc457df00eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.412037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc1674-cca5-4439-a2af-dc457df00eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.428311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-24cc1674-cca5-4439-a2af-dc457df00eb6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:24.429534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-24cc1674-cca5-4439-a2af-dc457df00eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:24.458028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:24.495043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 655/1294] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:24.517910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74e76d-a40c-4584-a87a-ab39a750434d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:24.548434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:24.621656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b74e76d-a40c-4584-a87a-ab39a750434d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:24.654492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429d924-4f9e-4c55-a563-a330e402f874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:24.655138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429d924-4f9e-4c55-a563-a330e402f874 None admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:24.655491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429d924-4f9e-4c55-a563-a330e402f874 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:24.655770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429d924-4f9e-4c55-a563-a330e402f874 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:24.657101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429d924-4f9e-4c55-a563-a330e402f874 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:24.658464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429d924-4f9e-4c55-a563-a330e402f874 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:24.674488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 640/1295] 10.222.0.43 () {60 vars in 1141 bytes} [Fri Jun 24 08:32:24 2022] POST /identity/v3/services => generated 246 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:32:26.690270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ad88-b460-4387-88f6-5fe02878144d None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:26.690848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ad88-b460-4387-88f6-5fe02878144d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:26.691115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992ad88-b460-4387-88f6-5fe02878144d None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:26.692534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 656/1296] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:26.697657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:26.698083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:26.698209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:26.714381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:26.715574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:26.785034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 641/1297] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:26.807858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992ad88-b460-4387-88f6-5fe02878144d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:26.912590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992ad88-b460-4387-88f6-5fe02878144d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:26.956511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:26.957243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:26.957561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:26.957839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:26.958455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:26.959878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:26.972795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:26.974242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:26.975253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 657/1298] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:32:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:26.980737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:26.999552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:27.079390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1ceba-986a-4ec3-8060-847df99a783e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:27.112583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:27.113235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:27.113516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:27.113791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:27.114488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:27.114794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:27.116185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:27.121188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 642/1299] 10.222.0.43 () {56 vars in 1116 bytes} [Fri Jun 24 08:32:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:27.126905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:27.157878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:27.224734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94183b66-7082-43eb-b167-1d57206f69aa None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:27.256920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:27.257582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:27.257867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:27.258145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:27.258944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] RBAC: Adding query filter params (interface=public, service_id=e6df6bbd7bcf4d11b852e83ba1013622, region_id=RegionOne) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:27.259283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:27.260669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4f152-d03f-491d-8fbf-db0aaaed4a72 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:27.266814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 658/1300] 10.222.0.43 () {56 vars in 1251 bytes} [Fri Jun 24 08:32:27 2022] GET /identity/v3/endpoints?service_id=e6df6bbd7bcf4d11b852e83ba1013622&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:27.272655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:27.352337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:27.369523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d362004-43e9-4bb4-9c4b-fb5f9482f3f0 None admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:27.401664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53de820-964a-4e80-8df6-288c00cd5c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:27.402309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53de820-964a-4e80-8df6-288c00cd5c87 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:27.402585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53de820-964a-4e80-8df6-288c00cd5c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:27.402857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53de820-964a-4e80-8df6-288c00cd5c87 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:27.403554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53de820-964a-4e80-8df6-288c00cd5c87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:27.404903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53de820-964a-4e80-8df6-288c00cd5c87 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:27.412052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 643/1301] 10.222.0.43 () {56 vars in 1087 bytes} [Fri Jun 24 08:32:27 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:32:29.408535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a730f-9cf8-4efe-ab7b-3932ccfd6a04 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:29.409080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a730f-9cf8-4efe-ab7b-3932ccfd6a04 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:29.409080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a730f-9cf8-4efe-ab7b-3932ccfd6a04 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:29.410476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 659/1302] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:32:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:32:29.415661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:29.416094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:29.416205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:29.433141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:32:29.434265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:32:29.498639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 644/1303] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:32:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 24 08:32:29.527602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a730f-9cf8-4efe-ab7b-3932ccfd6a04 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:29.621167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:29.628206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a730f-9cf8-4efe-ab7b-3932ccfd6a04 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:29.661413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:29.662199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:29.662484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:29.662764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:29.663438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:29.664801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 24 08:32:29.676253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 24 08:32:29.677432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.get_or_create( Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application with Lock( Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self._enter() Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application return self.creator() Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application created_value = creator( Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: ERROR keystone.server.flask.application Jun 24 08:32:29.678466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 660/1304] 10.222.0.43 () {56 vars in 1117 bytes} [Fri Jun 24 08:32:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 24 08:32:29.683491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:29.756835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:29.780527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e545cf-7132-4eb1-bc51-cc0672446f7c None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:29.813299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:29.813951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:29.814238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:29.814512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] PATH_INFO: `/v3/services` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:29.815234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:32:29.815546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:29.816936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df53dd64-1d5a-443b-b7e4-3164aea8de13 None admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:29.821945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 645/1305] 10.222.0.43 () {56 vars in 1116 bytes} [Fri Jun 24 08:32:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:32:29.827484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:32:29.935717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71053fa2-1095-482e-96fa-e0c78dc889aa None admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:32:29.969063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f069ad17-e874-4f97-945d-1a4d0ce7bb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:32:29.969717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069ad17-e874-4f97-945d-1a4d0ce7bb99 None admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:32:29.970000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069ad17-e874-4f97-945d-1a4d0ce7bb99 None admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:32:29.970278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f069ad17-e874-4f97-945d-1a4d0ce7bb99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:32:29.971022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069ad17-e874-4f97-945d-1a4d0ce7bb99 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:32:29.972408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f069ad17-e874-4f97-945d-1a4d0ce7bb99 None admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:32:29.989388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 661/1306] 10.222.0.43 () {60 vars in 1144 bytes} [Fri Jun 24 08:32:29 2022] POST /identity/v3/endpoints => generated 331 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:32:39.818951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=30 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:32:39.985486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:32.479376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964289-79d4-47e5-8a06-ea7e537c722b None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:32.480401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964289-79d4-47e5-8a06-ea7e537c722b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:32.480893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c964289-79d4-47e5-8a06-ea7e537c722b None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:32.488439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818fd8-434c-4723-b230-de3d1b05666e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:32.489347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818fd8-434c-4723-b230-de3d1b05666e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:32.489850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60818fd8-434c-4723-b230-de3d1b05666e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:32.493496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 646/1307] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:32 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:32.579833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-60818fd8-434c-4723-b230-de3d1b05666e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:32.580978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-60818fd8-434c-4723-b230-de3d1b05666e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:32.737284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 662/1308] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:33:32.808476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84827-64d9-4c9b-aa60-c4e082ffbcb5 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:32.808778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84827-64d9-4c9b-aa60-c4e082ffbcb5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:32.809043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84827-64d9-4c9b-aa60-c4e082ffbcb5 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:32.810665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 647/1309] 10.222.0.43 () {54 vars in 964 bytes} [Fri Jun 24 08:33:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:32.825702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db4ec-c5f9-407f-9df7-b2fea7b18f80 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:32.825999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db4ec-c5f9-407f-9df7-b2fea7b18f80 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:32.826288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684db4ec-c5f9-407f-9df7-b2fea7b18f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:32.996815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-684db4ec-c5f9-407f-9df7-b2fea7b18f80 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:32.998067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-684db4ec-c5f9-407f-9df7-b2fea7b18f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:33.046066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:33.131412 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 663/1310] 10.222.0.43 () {58 vars in 1061 bytes} [Fri Jun 24 08:33:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:33.140149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84827-64d9-4c9b-aa60-c4e082ffbcb5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:33.142362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-77a84827-64d9-4c9b-aa60-c4e082ffbcb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:33.266513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84827-64d9-4c9b-aa60-c4e082ffbcb5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:33.346418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:33.347079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:33.347390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:33.347672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:33.349339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:33.408611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:33.466443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:33.468117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908b461-4469-43b6-859c-aa27d54f2787 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:33.507697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 648/1311] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:35.844915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c693-597c-4a00-9465-ccde7e3140fc None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:35.845481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c693-597c-4a00-9465-ccde7e3140fc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:35.845481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27c693-597c-4a00-9465-ccde7e3140fc None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:35.846835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 664/1312] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:35.851934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78634005-0624-4670-bad3-8c1eb0cea77b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:35.852370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78634005-0624-4670-bad3-8c1eb0cea77b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:35.852503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78634005-0624-4670-bad3-8c1eb0cea77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:35.926591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-78634005-0624-4670-bad3-8c1eb0cea77b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:35.927836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-78634005-0624-4670-bad3-8c1eb0cea77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:36.026591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 649/1313] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:36.066101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c693-597c-4a00-9465-ccde7e3140fc None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:36.067736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fe27c693-597c-4a00-9465-ccde7e3140fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:36.180384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27c693-597c-4a00-9465-ccde7e3140fc None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:36.251078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:36.253995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:36.254759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:36.255233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:36.255688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:36.257514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:36.374520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:36.375997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa74b24-7a0f-40e8-9f61-b4c5c2ef173d service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:36.395638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 665/1314] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:38.699182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15118bb9-6272-4550-9753-a70fc34a30c3 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:38.699769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15118bb9-6272-4550-9753-a70fc34a30c3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:38.699769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15118bb9-6272-4550-9753-a70fc34a30c3 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:38.701149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 650/1315] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:38.706904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce977-6e64-4248-9871-fd439612acab None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:38.707340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce977-6e64-4248-9871-fd439612acab None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:38.707578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcce977-6e64-4248-9871-fd439612acab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:38.770445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6fcce977-6e64-4248-9871-fd439612acab None None] MFA Rules not processed for user `c33e8a8523ff46ea85b80ad1163fbde3`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:38.771674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6fcce977-6e64-4248-9871-fd439612acab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:38.822036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:38.866694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 666/1316] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:38.908462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15118bb9-6272-4550-9753-a70fc34a30c3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:38.909986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-15118bb9-6272-4550-9753-a70fc34a30c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:38.919810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:39.018519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15118bb9-6272-4550-9753-a70fc34a30c3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:39.088596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:39.089229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:39.089510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:39.089785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:39.091458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:39.152336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:39.199455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:39.200926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ac104-d95c-40f6-807b-ad654faf230f service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:39.218901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 651/1317] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:41.757008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ca7de-d525-4536-8b2d-4dd54fdcf9cb None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:41.757549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ca7de-d525-4536-8b2d-4dd54fdcf9cb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:41.757644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ca7de-d525-4536-8b2d-4dd54fdcf9cb None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:41.758979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 667/1318] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:33:41.764074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e536de-e01d-4aab-93f9-a0922d817455 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:41.764470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e536de-e01d-4aab-93f9-a0922d817455 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:41.764624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e536de-e01d-4aab-93f9-a0922d817455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:41.827431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-91e536de-e01d-4aab-93f9-a0922d817455 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:41.828628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-91e536de-e01d-4aab-93f9-a0922d817455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:41.927083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 652/1319] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:41 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:33:41.965184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ca7de-d525-4536-8b2d-4dd54fdcf9cb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:41.966692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-071ca7de-d525-4536-8b2d-4dd54fdcf9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:42.066373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ca7de-d525-4536-8b2d-4dd54fdcf9cb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:42.135908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:42.136548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:42.136829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:42.137104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:42.138702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:42.145820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:42.247266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:42.248701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ec2ee-66bf-4185-a544-1d749554eb68 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:42.268328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 668/1320] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:41 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:44.961263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047be81-b813-40db-8abd-c07f45bf018d None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:44.961808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047be81-b813-40db-8abd-c07f45bf018d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:44.961808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047be81-b813-40db-8abd-c07f45bf018d None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:44.963175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 653/1321] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:44.968942 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecb30b-a8ec-47e9-b271-5ab5d1158b60 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:44.969392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecb30b-a8ec-47e9-b271-5ab5d1158b60 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:44.969548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eecb30b-a8ec-47e9-b271-5ab5d1158b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.034114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2eecb30b-a8ec-47e9-b271-5ab5d1158b60 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:45.035350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2eecb30b-a8ec-47e9-b271-5ab5d1158b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:45.098633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:45.140562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 669/1322] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:45.178199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047be81-b813-40db-8abd-c07f45bf018d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:45.179804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2047be81-b813-40db-8abd-c07f45bf018d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:45.199859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:45.286014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047be81-b813-40db-8abd-c07f45bf018d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:45.357591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:45.358239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:45.358537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:45.358822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:45.360479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:45.430501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:45.468992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:45.470369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664268da-68a0-4436-a88c-3a0854194fdd service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:45.489989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 654/1323] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:47.581193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12ec78-5e79-407e-94e7-e3ef002f3f02 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:47.581766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12ec78-5e79-407e-94e7-e3ef002f3f02 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:47.581766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e12ec78-5e79-407e-94e7-e3ef002f3f02 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:47.583139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 670/1324] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:47.588608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b190c3-5909-4c93-aa99-06e291b7ba53 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:47.589044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b190c3-5909-4c93-aa99-06e291b7ba53 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:47.589295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b190c3-5909-4c93-aa99-06e291b7ba53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:47.652877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-65b190c3-5909-4c93-aa99-06e291b7ba53 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:47.654086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-65b190c3-5909-4c93-aa99-06e291b7ba53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:47.749301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 655/1325] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:47.785760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12ec78-5e79-407e-94e7-e3ef002f3f02 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:47.787305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1e12ec78-5e79-407e-94e7-e3ef002f3f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:47.896150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e12ec78-5e79-407e-94e7-e3ef002f3f02 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:47.972633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:47.973290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:47.973571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:47.973845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:47.975523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:47.992125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:48.085793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:48.087251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc20360-843c-4fd4-9515-4d51eaf9688b service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:48.113957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 671/1326] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:50.193629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bc9b5-6dda-495e-82a0-90b193e04b33 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:50.194144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bc9b5-6dda-495e-82a0-90b193e04b33 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:50.194144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97bc9b5-6dda-495e-82a0-90b193e04b33 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:50.195548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 656/1327] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:50.201048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd7714-0958-4f62-8258-eb1fb4bd9e7b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:50.201491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd7714-0958-4f62-8258-eb1fb4bd9e7b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:50.201751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd7714-0958-4f62-8258-eb1fb4bd9e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:50.268494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a4fd7714-0958-4f62-8258-eb1fb4bd9e7b None None] MFA Rules not processed for user `c33e8a8523ff46ea85b80ad1163fbde3`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:50.269711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a4fd7714-0958-4f62-8258-eb1fb4bd9e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:50.339799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:50.370904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 672/1328] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:50.409140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bc9b5-6dda-495e-82a0-90b193e04b33 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:50.410676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b97bc9b5-6dda-495e-82a0-90b193e04b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:50.441695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:50.515875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97bc9b5-6dda-495e-82a0-90b193e04b33 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:50.587944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:50.588599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:50.588884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:50.589160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:50.590799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:50.678479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:50.715774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:50.717245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93d54d-4aff-46d4-b1fe-69bfe6adf047 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:50.736792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 657/1329] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:53.011772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cadec-d4c7-4a5d-a41c-34d643c57e57 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:53.012316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cadec-d4c7-4a5d-a41c-34d643c57e57 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:53.012316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616cadec-d4c7-4a5d-a41c-34d643c57e57 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:53.013724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 673/1330] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:53.019492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d5514-af8b-42be-be47-7cf04d3d4dbe None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:53.019937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d5514-af8b-42be-be47-7cf04d3d4dbe None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:53.020033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d5514-af8b-42be-be47-7cf04d3d4dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:53.083707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4b9d5514-af8b-42be-be47-7cf04d3d4dbe None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:53.084885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4b9d5514-af8b-42be-be47-7cf04d3d4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:53.177592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 658/1331] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:53.225976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94396c69-be7b-4bd7-abb4-ca4d86bffe43 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:53.226263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94396c69-be7b-4bd7-abb4-ca4d86bffe43 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:53.226574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94396c69-be7b-4bd7-abb4-ca4d86bffe43 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:53.227973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 674/1332] 10.222.0.43 () {54 vars in 964 bytes} [Fri Jun 24 08:33:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:53.233776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852e77-2519-46f3-9512-295b551e6939 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:53.234170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852e77-2519-46f3-9512-295b551e6939 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:53.234454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97852e77-2519-46f3-9512-295b551e6939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:53.284378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:53.317099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-97852e77-2519-46f3-9512-295b551e6939 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:53.318325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-97852e77-2519-46f3-9512-295b551e6939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:53.392434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 659/1333] 10.222.0.43 () {58 vars in 1061 bytes} [Fri Jun 24 08:33:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:53.401619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94396c69-be7b-4bd7-abb4-ca4d86bffe43 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:53.403127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-94396c69-be7b-4bd7-abb4-ca4d86bffe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:53.504377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94396c69-be7b-4bd7-abb4-ca4d86bffe43 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:53.575200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:53.575890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:53.576177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:53.576457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:53.578127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:53.605071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:53.686318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:53.688345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6c113-e00f-4e40-b484-2655f8878543 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:53.705896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 675/1334] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:55.973199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6e30-27bb-4cbe-ac61-554de3a33517 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:55.973774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6e30-27bb-4cbe-ac61-554de3a33517 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:55.973774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db6e30-27bb-4cbe-ac61-554de3a33517 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:55.975118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 660/1335] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:55.980286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc623ce5-945b-4afd-bcd7-f0b4964305e6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:55.980699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc623ce5-945b-4afd-bcd7-f0b4964305e6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:55.980855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc623ce5-945b-4afd-bcd7-f0b4964305e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:56.044160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-fc623ce5-945b-4afd-bcd7-f0b4964305e6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:56.045351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fc623ce5-945b-4afd-bcd7-f0b4964305e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:56.118187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:56.142361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 676/1336] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:56.183443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db6e30-27bb-4cbe-ac61-554de3a33517 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:56.184986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d9db6e30-27bb-4cbe-ac61-554de3a33517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:56.286754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db6e30-27bb-4cbe-ac61-554de3a33517 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:56.314162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:56.357345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:56.357979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:56.358258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:56.359113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:56.360748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:56.468619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:56.470043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9c92c-750e-4d32-acc2-9c02320ead75 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:56.487852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 661/1337] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:33:59.120069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a76212-4e3e-4e41-8e8d-da412ab43df6 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:59.120671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a76212-4e3e-4e41-8e8d-da412ab43df6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:59.120671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a76212-4e3e-4e41-8e8d-da412ab43df6 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:59.121991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 677/1338] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:33:59.127599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45e1f2-225c-48d5-a616-7893eed98a88 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:59.128039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45e1f2-225c-48d5-a616-7893eed98a88 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:59.128145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45e1f2-225c-48d5-a616-7893eed98a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:59.191513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ca45e1f2-225c-48d5-a616-7893eed98a88 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:33:59.192717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ca45e1f2-225c-48d5-a616-7893eed98a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:59.201152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:59.297641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 662/1339] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:33:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:33:59.339531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a76212-4e3e-4e41-8e8d-da412ab43df6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:33:59.341051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-12a76212-4e3e-4e41-8e8d-da412ab43df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:59.444000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a76212-4e3e-4e41-8e8d-da412ab43df6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:33:59.518354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:33:59.519009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:33:59.519319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:33:59.519599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:33:59.521223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:33:59.556602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:33:59.630081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:33:59.631563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02013773-91bd-4c68-8958-3846449679e2 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:33:59.649593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 678/1340] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:33:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:03.694278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd0e4-331f-4765-aedd-a753bca3263b None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:03.694784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd0e4-331f-4765-aedd-a753bca3263b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:03.694784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd0e4-331f-4765-aedd-a753bca3263b None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:03.696141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 663/1341] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:03.701495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51d09-ee4e-4e1e-8199-364366fecef6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:03.701922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51d09-ee4e-4e1e-8199-364366fecef6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:03.702077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51d09-ee4e-4e1e-8199-364366fecef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:03.772653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-fdd51d09-ee4e-4e1e-8199-364366fecef6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:03.773846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fdd51d09-ee4e-4e1e-8199-364366fecef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:03.864216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:03.877436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 679/1342] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:34:03.917294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd0e4-331f-4765-aedd-a753bca3263b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:03.919561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f4fcd0e4-331f-4765-aedd-a753bca3263b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:04.021318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd0e4-331f-4765-aedd-a753bca3263b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:04.024767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c89f91c-970b-4a60-a9bf-b5764ad70f62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:04.025422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f91c-970b-4a60-a9bf-b5764ad70f62 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:04.025704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f91c-970b-4a60-a9bf-b5764ad70f62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:04.025980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89f91c-970b-4a60-a9bf-b5764ad70f62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:04.035736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89f91c-970b-4a60-a9bf-b5764ad70f62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:04.037173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c89f91c-970b-4a60-a9bf-b5764ad70f62 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:04.046581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 664/1343] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:34:06.690035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2306-ca6a-4e09-a3bc-af96e533764f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:06.690580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2306-ca6a-4e09-a3bc-af96e533764f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:06.690580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c2306-ca6a-4e09-a3bc-af96e533764f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:06.692011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 680/1344] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:06.697181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b51e7e-74a8-47de-9f49-39fd9a3d0695 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:06.697606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b51e7e-74a8-47de-9f49-39fd9a3d0695 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:06.697742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b51e7e-74a8-47de-9f49-39fd9a3d0695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:06.725174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-49b51e7e-74a8-47de-9f49-39fd9a3d0695 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:06.726352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-49b51e7e-74a8-47de-9f49-39fd9a3d0695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:06.758010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:06.812253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 665/1345] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:06.850887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2306-ca6a-4e09-a3bc-af96e533764f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:06.862719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c2306-ca6a-4e09-a3bc-af96e533764f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:06.866495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c199a2-f660-4cf4-a68c-4848dc618a02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:06.867157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c199a2-f660-4cf4-a68c-4848dc618a02 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:06.867470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c199a2-f660-4cf4-a68c-4848dc618a02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:06.867749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c199a2-f660-4cf4-a68c-4848dc618a02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:06.877523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c199a2-f660-4cf4-a68c-4848dc618a02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:06.878977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c199a2-f660-4cf4-a68c-4848dc618a02 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:06.890696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 681/1346] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:10.850865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc4e17-0eae-45b7-b9a6-c3e723e8573a None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:10.851462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc4e17-0eae-45b7-b9a6-c3e723e8573a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:10.851462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc4e17-0eae-45b7-b9a6-c3e723e8573a None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:10.852840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 666/1347] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:10.857939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f7703-973e-475b-b6ff-7ade052e693f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:10.858842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f7703-973e-475b-b6ff-7ade052e693f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:10.859032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f7703-973e-475b-b6ff-7ade052e693f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:10.876861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-cf7f7703-973e-475b-b6ff-7ade052e693f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:10.878067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cf7f7703-973e-475b-b6ff-7ade052e693f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:10.887791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 682/1348] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:10.926624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc4e17-0eae-45b7-b9a6-c3e723e8573a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:10.938190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc4e17-0eae-45b7-b9a6-c3e723e8573a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:10.941841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdeb1ed4-6539-4faf-aecd-487648bef7a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:10.943399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb1ed4-6539-4faf-aecd-487648bef7a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:10.944086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb1ed4-6539-4faf-aecd-487648bef7a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:10.944436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdeb1ed4-6539-4faf-aecd-487648bef7a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:10.954368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb1ed4-6539-4faf-aecd-487648bef7a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:10.956120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdeb1ed4-6539-4faf-aecd-487648bef7a8 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:10.965723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 667/1349] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:15.997529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c10b6-a4a8-4569-8db7-aa401873b4bc None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:15.998039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c10b6-a4a8-4569-8db7-aa401873b4bc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:15.998156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9c10b6-a4a8-4569-8db7-aa401873b4bc None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:16.008244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 683/1350] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:34:15 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:16.013985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f936-8dac-4b70-8413-ec5fcf211cf3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:16.014991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f936-8dac-4b70-8413-ec5fcf211cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:16.016169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09f936-8dac-4b70-8413-ec5fcf211cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:16.106796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6a09f936-8dac-4b70-8413-ec5fcf211cf3 None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:16.108026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6a09f936-8dac-4b70-8413-ec5fcf211cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:16.191878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 668/1351] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:34:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:20.330121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78063a-87f4-4e3b-b240-92ae37f3abf2 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:20.330658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78063a-87f4-4e3b-b240-92ae37f3abf2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:20.330658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be78063a-87f4-4e3b-b240-92ae37f3abf2 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:20.332330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 684/1352] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:34:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:20.337365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57305637-9ee0-48b3-988a-57baf6cb95c1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:20.337787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57305637-9ee0-48b3-988a-57baf6cb95c1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:20.337989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57305637-9ee0-48b3-988a-57baf6cb95c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:20.357498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-57305637-9ee0-48b3-988a-57baf6cb95c1 None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:20.359628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-57305637-9ee0-48b3-988a-57baf6cb95c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:20.368571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 669/1353] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:34:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:20.871314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=13 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:21.834357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac52746-8511-42f2-b196-379c426c667b None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:21.835042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac52746-8511-42f2-b196-379c426c667b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:21.835042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac52746-8511-42f2-b196-379c426c667b None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:21.836566 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 685/1354] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:34:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:21.841806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71893a9-36a2-4c8b-9ce3-6200422d0ecd None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:21.842243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71893a9-36a2-4c8b-9ce3-6200422d0ecd None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:21.842888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71893a9-36a2-4c8b-9ce3-6200422d0ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:21.861134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a71893a9-36a2-4c8b-9ce3-6200422d0ecd None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:21.862336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a71893a9-36a2-4c8b-9ce3-6200422d0ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:21.870931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 670/1355] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:34:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:25.629213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461d20f-b9ec-41b9-9043-e4f7513e7a54 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.629775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461d20f-b9ec-41b9-9043-e4f7513e7a54 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.629775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0461d20f-b9ec-41b9-9043-e4f7513e7a54 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.631153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 686/1356] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:25.636588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824851eb-1d71-42ea-ad7c-a3d31e4d4a9d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.637032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824851eb-1d71-42ea-ad7c-a3d31e4d4a9d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.637115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824851eb-1d71-42ea-ad7c-a3d31e4d4a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.658335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-824851eb-1d71-42ea-ad7c-a3d31e4d4a9d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:25.659789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-824851eb-1d71-42ea-ad7c-a3d31e4d4a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:25.668594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 671/1357] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:25.726641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e1b39-216e-43dd-bf60-989eec4de204 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.727588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e1b39-216e-43dd-bf60-989eec4de204 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.728090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e1b39-216e-43dd-bf60-989eec4de204 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.730179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 687/1358] 10.222.0.43 () {54 vars in 961 bytes} [Fri Jun 24 08:34:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:25.738992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6d0c8-c1cf-47d1-9db2-b00a275aa923 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.739308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6d0c8-c1cf-47d1-9db2-b00a275aa923 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.739588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb6d0c8-c1cf-47d1-9db2-b00a275aa923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.789381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:25.798954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2cb6d0c8-c1cf-47d1-9db2-b00a275aa923 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:25.800304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2cb6d0c8-c1cf-47d1-9db2-b00a275aa923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:25.895522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 672/1359] 10.222.0.43 () {58 vars in 1058 bytes} [Fri Jun 24 08:34:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:25.907786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e1b39-216e-43dd-bf60-989eec4de204 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:25.926619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e1b39-216e-43dd-bf60-989eec4de204 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:25.932260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13cb116-25c0-4e28-af84-4f03df1fa771 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:25.933016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb116-25c0-4e28-af84-4f03df1fa771 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:25.933404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb116-25c0-4e28-af84-4f03df1fa771 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:25.933811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13cb116-25c0-4e28-af84-4f03df1fa771 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:25.943619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cb116-25c0-4e28-af84-4f03df1fa771 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:25.945337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13cb116-25c0-4e28-af84-4f03df1fa771 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:25.955838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 688/1360] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:28.186047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb61adb-5bf8-44a8-be1f-e13095eaf42f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:28.186548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb61adb-5bf8-44a8-be1f-e13095eaf42f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:28.186645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb61adb-5bf8-44a8-be1f-e13095eaf42f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:28.189724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 673/1361] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:28.193164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7458bd-6796-4d86-8d6d-0c7729d8c101 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:28.193971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7458bd-6796-4d86-8d6d-0c7729d8c101 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:28.194251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7458bd-6796-4d86-8d6d-0c7729d8c101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:28.213187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-ba7458bd-6796-4d86-8d6d-0c7729d8c101 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:28.214378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ba7458bd-6796-4d86-8d6d-0c7729d8c101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:28.222630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 689/1362] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:28.265977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb61adb-5bf8-44a8-be1f-e13095eaf42f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:28.281438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb61adb-5bf8-44a8-be1f-e13095eaf42f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:28.287070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18674c4a-27a9-4aa1-a1c9-0940db630337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:28.287843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674c4a-27a9-4aa1-a1c9-0940db630337 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:28.288222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674c4a-27a9-4aa1-a1c9-0940db630337 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:28.288594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18674c4a-27a9-4aa1-a1c9-0940db630337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:28.297199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674c4a-27a9-4aa1-a1c9-0940db630337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:28.298877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18674c4a-27a9-4aa1-a1c9-0940db630337 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:28.308992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 674/1363] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:30.670984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806d358-d907-4023-b45c-7d40d0975028 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.671801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806d358-d907-4023-b45c-7d40d0975028 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.671801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b806d358-d907-4023-b45c-7d40d0975028 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.673097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 690/1364] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:30.678686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bb3a2-544d-4f4d-a64c-b4ee0531bc5d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.679123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bb3a2-544d-4f4d-a64c-b4ee0531bc5d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.679292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520bb3a2-544d-4f4d-a64c-b4ee0531bc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.715377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-520bb3a2-544d-4f4d-a64c-b4ee0531bc5d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:30.716757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-520bb3a2-544d-4f4d-a64c-b4ee0531bc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:30.725217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 675/1365] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:34:30.771995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabce54-5e07-4ef3-bcba-c5f3b8a1b29d None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.772311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabce54-5e07-4ef3-bcba-c5f3b8a1b29d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.772588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabce54-5e07-4ef3-bcba-c5f3b8a1b29d None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.773987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 691/1366] 10.222.0.43 () {54 vars in 961 bytes} [Fri Jun 24 08:34:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:30.779726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642c8d4-c133-47cb-bd2d-1b828ad0e156 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.780022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642c8d4-c133-47cb-bd2d-1b828ad0e156 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.780325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642c8d4-c133-47cb-bd2d-1b828ad0e156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.798197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1642c8d4-c133-47cb-bd2d-1b828ad0e156 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:30.799441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1642c8d4-c133-47cb-bd2d-1b828ad0e156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:30.808821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 676/1367] 10.222.0.43 () {58 vars in 1058 bytes} [Fri Jun 24 08:34:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:30.816860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabce54-5e07-4ef3-bcba-c5f3b8a1b29d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:30.831958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabce54-5e07-4ef3-bcba-c5f3b8a1b29d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:30.837384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1280a-d678-4037-b41b-7ac61528cb20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:30.838195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1280a-d678-4037-b41b-7ac61528cb20 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:30.838482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1280a-d678-4037-b41b-7ac61528cb20 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:30.838762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1280a-d678-4037-b41b-7ac61528cb20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:30.846800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1280a-d678-4037-b41b-7ac61528cb20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:30.848410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f1280a-d678-4037-b41b-7ac61528cb20 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:30.857712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 692/1368] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:32.947969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd659fe9-12af-4459-9618-1e618d51efd3 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:32.948529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd659fe9-12af-4459-9618-1e618d51efd3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:32.948529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd659fe9-12af-4459-9618-1e618d51efd3 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:32.950047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 677/1369] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:32.955929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1ac1b-fcc5-47f4-8096-d0295c1c5ea0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:32.956697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1ac1b-fcc5-47f4-8096-d0295c1c5ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:32.956980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d1ac1b-fcc5-47f4-8096-d0295c1c5ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:32.974749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-46d1ac1b-fcc5-47f4-8096-d0295c1c5ea0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:32.976016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-46d1ac1b-fcc5-47f4-8096-d0295c1c5ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:32.984344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 693/1370] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:33.030427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd659fe9-12af-4459-9618-1e618d51efd3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:33.046294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd659fe9-12af-4459-9618-1e618d51efd3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:33.052138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4859c5fd-de6c-4013-b7bc-c865188e0f92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:33.052778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c5fd-de6c-4013-b7bc-c865188e0f92 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:33.053058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c5fd-de6c-4013-b7bc-c865188e0f92 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:33.053329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4859c5fd-de6c-4013-b7bc-c865188e0f92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:33.062381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859c5fd-de6c-4013-b7bc-c865188e0f92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:33.064145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4859c5fd-de6c-4013-b7bc-c865188e0f92 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:33.074548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 678/1371] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:33.715319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd2df-e1aa-4cb8-8813-fb8b741eea27 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:33.716066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd2df-e1aa-4cb8-8813-fb8b741eea27 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:33.716570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fdd2df-e1aa-4cb8-8813-fb8b741eea27 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:33.720678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 694/1372] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:33.724096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346ea73-c771-43cf-9574-c48fa4e31964 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:33.724387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346ea73-c771-43cf-9574-c48fa4e31964 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:33.724665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3346ea73-c771-43cf-9574-c48fa4e31964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:33.743136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3346ea73-c771-43cf-9574-c48fa4e31964 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:33.744368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3346ea73-c771-43cf-9574-c48fa4e31964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:33.752535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 679/1373] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:34.095904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164c64-d335-4ecc-9f23-6c08fa12cbcd None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:34.095904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164c64-d335-4ecc-9f23-6c08fa12cbcd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:34.095904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04164c64-d335-4ecc-9f23-6c08fa12cbcd None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:34.097970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 695/1374] 10.222.0.43 () {54 vars in 963 bytes} [Fri Jun 24 08:34:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:34.101496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7a5c-1daf-45d7-9c1f-36873a962686 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:34.102043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7a5c-1daf-45d7-9c1f-36873a962686 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:34.102043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51c7a5c-1daf-45d7-9c1f-36873a962686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:34.152547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e51c7a5c-1daf-45d7-9c1f-36873a962686 None None] MFA Rules not processed for user `f3229f7fdd344dda97a47709c93ce2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:34.153758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e51c7a5c-1daf-45d7-9c1f-36873a962686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:34.227032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 680/1375] 10.222.0.43 () {58 vars in 1060 bytes} [Fri Jun 24 08:34:34 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:34.234799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04164c64-d335-4ecc-9f23-6c08fa12cbcd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:34.246664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04164c64-d335-4ecc-9f23-6c08fa12cbcd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:34.264873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a0d268-e551-418c-8d9b-35491c9a393f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:34.266125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0d268-e551-418c-8d9b-35491c9a393f service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:34.266756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0d268-e551-418c-8d9b-35491c9a393f service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:34.267419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a0d268-e551-418c-8d9b-35491c9a393f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:34.282958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0d268-e551-418c-8d9b-35491c9a393f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:34.284986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a0d268-e551-418c-8d9b-35491c9a393f service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:34.296578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 696/1376] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:35.141154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926d358-a807-4cce-8e9b-9a33785b8679 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:35.141708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926d358-a807-4cce-8e9b-9a33785b8679 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:35.141832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926d358-a807-4cce-8e9b-9a33785b8679 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:35.143107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 681/1377] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:35.148618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458967c-08fe-41da-a94a-ffeaa8a059d7 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:35.149028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458967c-08fe-41da-a94a-ffeaa8a059d7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:35.149294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3458967c-08fe-41da-a94a-ffeaa8a059d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:35.167621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3458967c-08fe-41da-a94a-ffeaa8a059d7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:35.168884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3458967c-08fe-41da-a94a-ffeaa8a059d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:35.177657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 697/1378] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:35.221509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926d358-a807-4cce-8e9b-9a33785b8679 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:35.241259 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926d358-a807-4cce-8e9b-9a33785b8679 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:35.246745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b87ab2-342c-40f6-92d3-e0ead7a0be6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:35.247440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b87ab2-342c-40f6-92d3-e0ead7a0be6e service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:35.248317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b87ab2-342c-40f6-92d3-e0ead7a0be6e service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:35.248604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b87ab2-342c-40f6-92d3-e0ead7a0be6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:35.256686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b87ab2-342c-40f6-92d3-e0ead7a0be6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:35.258264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b87ab2-342c-40f6-92d3-e0ead7a0be6e service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:35.262189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:35.268797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 682/1379] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:37.328641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516fd5c-2ecd-4c28-a273-8a52d070d834 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:37.329184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516fd5c-2ecd-4c28-a273-8a52d070d834 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:37.329184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d516fd5c-2ecd-4c28-a273-8a52d070d834 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:37.331717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 698/1380] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:37.337209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf171-d2c2-4650-b5fb-8c5e8b8db8a5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:37.337653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf171-d2c2-4650-b5fb-8c5e8b8db8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:37.337754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1cf171-d2c2-4650-b5fb-8c5e8b8db8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:37.361703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5c1cf171-d2c2-4650-b5fb-8c5e8b8db8a5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:37.363065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5c1cf171-d2c2-4650-b5fb-8c5e8b8db8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:37.371650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 683/1381] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:37.416560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516fd5c-2ecd-4c28-a273-8a52d070d834 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:37.431431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d516fd5c-2ecd-4c28-a273-8a52d070d834 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:37.437517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e1270-280c-441f-be68-4d320230e0e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:37.438160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1270-280c-441f-be68-4d320230e0e6 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:37.438445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1270-280c-441f-be68-4d320230e0e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:37.438726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e1270-280c-441f-be68-4d320230e0e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:37.446902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e1270-280c-441f-be68-4d320230e0e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:37.448517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e1270-280c-441f-be68-4d320230e0e6 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:37.458501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 699/1382] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:39.525348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc746c-09b6-4afc-ad4f-05fb441b1ee0 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:39.525941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc746c-09b6-4afc-ad4f-05fb441b1ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:39.526292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bc746c-09b6-4afc-ad4f-05fb441b1ee0 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:39.529709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 684/1383] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:39.537665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86961da-986c-42f0-b290-b67457e9f3cd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:39.538118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86961da-986c-42f0-b290-b67457e9f3cd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:39.538199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86961da-986c-42f0-b290-b67457e9f3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:39.556204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c86961da-986c-42f0-b290-b67457e9f3cd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:39.557378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c86961da-986c-42f0-b290-b67457e9f3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:39.566081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 700/1384] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:39.609197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc746c-09b6-4afc-ad4f-05fb441b1ee0 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:39.623743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bc746c-09b6-4afc-ad4f-05fb441b1ee0 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:39.629283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c86e5fd-03d1-4222-83e2-5ca0a84da0b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:39.629900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86e5fd-03d1-4222-83e2-5ca0a84da0b0 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:39.630178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86e5fd-03d1-4222-83e2-5ca0a84da0b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:39.630627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c86e5fd-03d1-4222-83e2-5ca0a84da0b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:39.639098 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86e5fd-03d1-4222-83e2-5ca0a84da0b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:39.640708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c86e5fd-03d1-4222-83e2-5ca0a84da0b0 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:39.650465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 685/1385] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:41.713848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4eb8-bd72-4914-b30f-894de03e84db None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:41.713848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4eb8-bd72-4914-b30f-894de03e84db None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:41.714697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d4eb8-bd72-4914-b30f-894de03e84db None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:41.715748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 701/1386] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:41.721252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b6afd-0b5c-4492-a3f9-e2b51be92489 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:41.721966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b6afd-0b5c-4492-a3f9-e2b51be92489 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:41.721966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b6afd-0b5c-4492-a3f9-e2b51be92489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:41.739624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7e9b6afd-0b5c-4492-a3f9-e2b51be92489 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:41.740933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7e9b6afd-0b5c-4492-a3f9-e2b51be92489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:41.762824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 686/1387] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:41.793235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d4eb8-bd72-4914-b30f-894de03e84db None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:41.811043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d4eb8-bd72-4914-b30f-894de03e84db None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:41.817414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018f7bb-5d68-4879-b7c4-aaa05f3692fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:41.819118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018f7bb-5d68-4879-b7c4-aaa05f3692fa service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:41.819540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018f7bb-5d68-4879-b7c4-aaa05f3692fa service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:41.819886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018f7bb-5d68-4879-b7c4-aaa05f3692fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:41.829936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018f7bb-5d68-4879-b7c4-aaa05f3692fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:41.832322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018f7bb-5d68-4879-b7c4-aaa05f3692fa service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:41.844345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 702/1388] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:44.204908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413f16-9754-42fc-ab8b-957cb71bf774 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:44.205435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413f16-9754-42fc-ab8b-957cb71bf774 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:44.205646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b413f16-9754-42fc-ab8b-957cb71bf774 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:44.207543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 687/1389] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:44.212757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dbb8b-45c6-4124-a0b9-dc6a428cd1ac None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:44.213164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dbb8b-45c6-4124-a0b9-dc6a428cd1ac None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:44.213395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381dbb8b-45c6-4124-a0b9-dc6a428cd1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:44.231264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-381dbb8b-45c6-4124-a0b9-dc6a428cd1ac None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:44.232503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-381dbb8b-45c6-4124-a0b9-dc6a428cd1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:44.240601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 703/1390] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:34:44.287118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413f16-9754-42fc-ab8b-957cb71bf774 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:44.301489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b413f16-9754-42fc-ab8b-957cb71bf774 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:44.307035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c8190-0df0-4abf-96d0-6e9e5f969da4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:44.307706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c8190-0df0-4abf-96d0-6e9e5f969da4 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:44.307993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c8190-0df0-4abf-96d0-6e9e5f969da4 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:44.308270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c8190-0df0-4abf-96d0-6e9e5f969da4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:44.317391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c8190-0df0-4abf-96d0-6e9e5f969da4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:44.318977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c8190-0df0-4abf-96d0-6e9e5f969da4 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:44.329592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 688/1391] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:46.406703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234559e-c473-464f-94b7-bb095ea959a7 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:46.407421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234559e-c473-464f-94b7-bb095ea959a7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:46.407421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3234559e-c473-464f-94b7-bb095ea959a7 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:46.408778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 704/1392] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:46.414045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619adb-d597-4b7b-9f56-603255603322 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:46.414470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619adb-d597-4b7b-9f56-603255603322 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:46.414607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52619adb-d597-4b7b-9f56-603255603322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:46.432156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-52619adb-d597-4b7b-9f56-603255603322 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:46.433284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-52619adb-d597-4b7b-9f56-603255603322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:46.441611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 689/1393] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:46.485712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234559e-c473-464f-94b7-bb095ea959a7 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:46.504255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3234559e-c473-464f-94b7-bb095ea959a7 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:46.511416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d009ef-21e7-4611-a27d-cb44ad226b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:46.512207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d009ef-21e7-4611-a27d-cb44ad226b40 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:46.512567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d009ef-21e7-4611-a27d-cb44ad226b40 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:46.512928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d009ef-21e7-4611-a27d-cb44ad226b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:46.523727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d009ef-21e7-4611-a27d-cb44ad226b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:46.525787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d009ef-21e7-4611-a27d-cb44ad226b40 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:46.538903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 705/1394] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:48.596274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c348c-0018-44cc-b54d-28b6b763b55e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:48.596849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c348c-0018-44cc-b54d-28b6b763b55e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:48.596849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960c348c-0018-44cc-b54d-28b6b763b55e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:48.598213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 690/1395] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:48.603350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3989f7-3d39-4c21-a660-57f89e9e3aac None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:48.603812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3989f7-3d39-4c21-a660-57f89e9e3aac None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:48.603918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3989f7-3d39-4c21-a660-57f89e9e3aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:48.622215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-de3989f7-3d39-4c21-a660-57f89e9e3aac None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:48.623471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-de3989f7-3d39-4c21-a660-57f89e9e3aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:48.631284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 706/1396] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:48.675419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c348c-0018-44cc-b54d-28b6b763b55e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:48.689893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960c348c-0018-44cc-b54d-28b6b763b55e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:48.695132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58cf90f-c210-4aa0-8f97-12edc4357535 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:48.695806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cf90f-c210-4aa0-8f97-12edc4357535 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:48.696086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cf90f-c210-4aa0-8f97-12edc4357535 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:48.696361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58cf90f-c210-4aa0-8f97-12edc4357535 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:48.704416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cf90f-c210-4aa0-8f97-12edc4357535 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:48.705944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58cf90f-c210-4aa0-8f97-12edc4357535 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:48.715368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 691/1397] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:50.121344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad45b3-9a3c-40fa-9d75-0468e09ece07 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.121881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad45b3-9a3c-40fa-9d75-0468e09ece07 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.121972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad45b3-9a3c-40fa-9d75-0468e09ece07 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.123331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 707/1398] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:50.128609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069dbfd-4a47-4f00-85ef-4c11816e0738 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.129012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069dbfd-4a47-4f00-85ef-4c11816e0738 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.129172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0069dbfd-4a47-4f00-85ef-4c11816e0738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.146677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-0069dbfd-4a47-4f00-85ef-4c11816e0738 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:50.147876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0069dbfd-4a47-4f00-85ef-4c11816e0738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:50.156385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 692/1399] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:50.212139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a83ca-8f87-4d78-b705-ccd6df3141ba None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.212500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a83ca-8f87-4d78-b705-ccd6df3141ba None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.212848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a83ca-8f87-4d78-b705-ccd6df3141ba None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.214224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 708/1400] 10.222.0.43 () {54 vars in 962 bytes} [Fri Jun 24 08:34:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:50.219884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a6b75-f643-44db-87be-c5da5c6425e5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.220243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a6b75-f643-44db-87be-c5da5c6425e5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.220591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a6b75-f643-44db-87be-c5da5c6425e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.267815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-997a6b75-f643-44db-87be-c5da5c6425e5 None None] MFA Rules not processed for user `ffea41aca5bd4fd49e2c5f74ac317814`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:50.269037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-997a6b75-f643-44db-87be-c5da5c6425e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:50.328095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 693/1401] 10.222.0.43 () {58 vars in 1059 bytes} [Fri Jun 24 08:34:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:50.335606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a83ca-8f87-4d78-b705-ccd6df3141ba None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:50.346975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a83ca-8f87-4d78-b705-ccd6df3141ba None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:50.350468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efdf087-09a3-48a8-874c-ebaea6263fc0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:50.351108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdf087-09a3-48a8-874c-ebaea6263fc0 service glance] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.351429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdf087-09a3-48a8-874c-ebaea6263fc0 service glance] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.351710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efdf087-09a3-48a8-874c-ebaea6263fc0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.361284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdf087-09a3-48a8-874c-ebaea6263fc0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:50.362701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efdf087-09a3-48a8-874c-ebaea6263fc0 service glance] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:50.372219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 709/1402] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:50.762284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3c05-f4e0-4144-8525-a089a5139a13 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.762626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3c05-f4e0-4144-8525-a089a5139a13 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.762916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3c05-f4e0-4144-8525-a089a5139a13 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.764924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 694/1403] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:50.770169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff0190-c328-450f-b685-c70eaed053b4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.770465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff0190-c328-450f-b685-c70eaed053b4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.770743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ff0190-c328-450f-b685-c70eaed053b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.789215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-76ff0190-c328-450f-b685-c70eaed053b4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:50.790381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-76ff0190-c328-450f-b685-c70eaed053b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:50.798078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 710/1404] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:34:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:50.842677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a3c05-f4e0-4144-8525-a089a5139a13 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:50.857573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a3c05-f4e0-4144-8525-a089a5139a13 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:50.863075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb6e52-705e-44ff-a42e-60d712a6a354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:50.863981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb6e52-705e-44ff-a42e-60d712a6a354 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:50.864262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb6e52-705e-44ff-a42e-60d712a6a354 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:50.864567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb6e52-705e-44ff-a42e-60d712a6a354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:50.873871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb6e52-705e-44ff-a42e-60d712a6a354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:50.875598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb6e52-705e-44ff-a42e-60d712a6a354 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:50.884861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 695/1405] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:34:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:34:51.093010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574b98-f0e9-4162-a7e2-4ca3ffd9e8a3 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:51.093273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574b98-f0e9-4162-a7e2-4ca3ffd9e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:51.093558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8574b98-f0e9-4162-a7e2-4ca3ffd9e8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:51.098794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=64 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:34:51.145164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c8574b98-f0e9-4162-a7e2-4ca3ffd9e8a3 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:51.146361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c8574b98-f0e9-4162-a7e2-4ca3ffd9e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:51.210225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 711/1406] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:34:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:51.254592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0441d1-3367-438b-bd57-4d1a7a908c86 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:51.255066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0441d1-3367-438b-bd57-4d1a7a908c86 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:51.255164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0441d1-3367-438b-bd57-4d1a7a908c86 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:51.256575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 696/1407] 10.222.0.43 () {54 vars in 942 bytes} [Fri Jun 24 08:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:34:51.262694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab196745-757f-4600-a63a-2665573c717b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:51.262984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab196745-757f-4600-a63a-2665573c717b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:51.263293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab196745-757f-4600-a63a-2665573c717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:51.311298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-ab196745-757f-4600-a63a-2665573c717b None None] MFA Rules not processed for user `69edde6d461545069a9e98c661e5453e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:34:51.312508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ab196745-757f-4600-a63a-2665573c717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:34:51.382007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 712/1408] 10.222.0.43 () {58 vars in 1039 bytes} [Fri Jun 24 08:34:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:34:51.391395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0441d1-3367-438b-bd57-4d1a7a908c86 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:34:51.404878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0441d1-3367-438b-bd57-4d1a7a908c86 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:34:51.409723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98146095-5bff-4552-9cf0-45a4917502b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:34:51.410358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146095-5bff-4552-9cf0-45a4917502b8 service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:34:51.410646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146095-5bff-4552-9cf0-45a4917502b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:34:51.410924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98146095-5bff-4552-9cf0-45a4917502b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:34:51.418706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98146095-5bff-4552-9cf0-45a4917502b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:34:51.420828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98146095-5bff-4552-9cf0-45a4917502b8 service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:34:51.428685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 697/1409] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:34:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:35:01.379203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=83 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:01.425504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=81 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:13.510083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedea6b-3f33-43db-b8e3-f324cf89c228 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.510616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedea6b-3f33-43db-b8e3-f324cf89c228 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.510616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdedea6b-3f33-43db-b8e3-f324cf89c228 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.511981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 713/1410] 10.222.0.43 () {54 vars in 918 bytes} [Fri Jun 24 08:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:13.517423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd1091-9b5d-454f-b060-0e440474146d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.517847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd1091-9b5d-454f-b060-0e440474146d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.517982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddd1091-9b5d-454f-b060-0e440474146d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.536016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3ddd1091-9b5d-454f-b060-0e440474146d None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:13.537190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3ddd1091-9b5d-454f-b060-0e440474146d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:13.545441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 698/1411] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:13.558902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8df15-6347-462e-a10d-46f192512cb0 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.559194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8df15-6347-462e-a10d-46f192512cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.559510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe8df15-6347-462e-a10d-46f192512cb0 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.561332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 714/1412] 10.222.0.43 () {54 vars in 918 bytes} [Fri Jun 24 08:35:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:13.565598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb33234-90c8-423e-b2ee-8b4974997177 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:13.565885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb33234-90c8-423e-b2ee-8b4974997177 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:13.566164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb33234-90c8-423e-b2ee-8b4974997177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:13.583925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5bb33234-90c8-423e-b2ee-8b4974997177 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:13.585098 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5bb33234-90c8-423e-b2ee-8b4974997177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:13.594124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 699/1413] 10.222.0.43 () {58 vars in 1015 bytes} [Fri Jun 24 08:35:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:16.063003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf2249-955a-4a7c-af5a-5b1a4ec41e35 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:16.063528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf2249-955a-4a7c-af5a-5b1a4ec41e35 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:16.063631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbf2249-955a-4a7c-af5a-5b1a4ec41e35 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:16.064970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 715/1414] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:16.070326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc48c-1d48-47e6-a429-15ce68752cf2 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:16.070741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc48c-1d48-47e6-a429-15ce68752cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:16.070894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdc48c-1d48-47e6-a429-15ce68752cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:16.088326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f4fdc48c-1d48-47e6-a429-15ce68752cf2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:16.089498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f4fdc48c-1d48-47e6-a429-15ce68752cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:16.097278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 700/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:16.140562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf2249-955a-4a7c-af5a-5b1a4ec41e35 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:16.155068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbf2249-955a-4a7c-af5a-5b1a4ec41e35 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:16.160352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346ff9ac-0cda-4389-9284-e858df13f4e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:16.160978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ff9ac-0cda-4389-9284-e858df13f4e3 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:16.161257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ff9ac-0cda-4389-9284-e858df13f4e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:16.161528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346ff9ac-0cda-4389-9284-e858df13f4e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:16.170180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ff9ac-0cda-4389-9284-e858df13f4e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:16.171825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346ff9ac-0cda-4389-9284-e858df13f4e3 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:16.181493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 716/1416] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:17.071297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5855e3-90f8-4b62-b610-3dc567ce0927 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:17.071875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5855e3-90f8-4b62-b610-3dc567ce0927 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:17.071875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5855e3-90f8-4b62-b610-3dc567ce0927 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:17.073249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 701/1417] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:17.078577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a42656-9964-46aa-8b75-317d1161a415 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:17.079086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a42656-9964-46aa-8b75-317d1161a415 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:17.079203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a42656-9964-46aa-8b75-317d1161a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:17.097446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-17a42656-9964-46aa-8b75-317d1161a415 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:17.098634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-17a42656-9964-46aa-8b75-317d1161a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:17.106359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 717/1418] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:17.255547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5855e3-90f8-4b62-b610-3dc567ce0927 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:17.270185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5855e3-90f8-4b62-b610-3dc567ce0927 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:17.275679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaf7b97-c114-4d87-8285-16f5eb913987 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:17.276316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7b97-c114-4d87-8285-16f5eb913987 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:17.276603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7b97-c114-4d87-8285-16f5eb913987 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:17.276882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaf7b97-c114-4d87-8285-16f5eb913987 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:17.285481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf7b97-c114-4d87-8285-16f5eb913987 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:17.287077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaf7b97-c114-4d87-8285-16f5eb913987 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:17.296438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 702/1419] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:18.181717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a948aee-8da0-4b98-9a04-03a75029fb02 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:18.181717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a948aee-8da0-4b98-9a04-03a75029fb02 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:18.181717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a948aee-8da0-4b98-9a04-03a75029fb02 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:18.182966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 718/1420] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:18.188546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1af98-372f-4935-a595-e949edf14560 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:18.189017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1af98-372f-4935-a595-e949edf14560 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:18.189122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1af98-372f-4935-a595-e949edf14560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:18.207423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6da1af98-372f-4935-a595-e949edf14560 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:18.208616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6da1af98-372f-4935-a595-e949edf14560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:18.216327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 703/1421] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:18.366864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a948aee-8da0-4b98-9a04-03a75029fb02 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:18.381552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a948aee-8da0-4b98-9a04-03a75029fb02 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:18.386837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec856791-4eba-4462-9693-70aaf503c4e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:18.387508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec856791-4eba-4462-9693-70aaf503c4e5 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:18.387790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec856791-4eba-4462-9693-70aaf503c4e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:18.388066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec856791-4eba-4462-9693-70aaf503c4e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:18.396541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec856791-4eba-4462-9693-70aaf503c4e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:18.398109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec856791-4eba-4462-9693-70aaf503c4e5 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:18.408281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 719/1422] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:19.261597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b3440-234a-47c3-8bb1-ac54f5c92852 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:19.262149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b3440-234a-47c3-8bb1-ac54f5c92852 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:19.262149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b3440-234a-47c3-8bb1-ac54f5c92852 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:19.263580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 704/1423] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:19.268579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b98a-1a95-46c4-927c-8251f5161d1c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:19.268983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b98a-1a95-46c4-927c-8251f5161d1c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:19.269137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b98a-1a95-46c4-927c-8251f5161d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:19.286849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8d88b98a-1a95-46c4-927c-8251f5161d1c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:19.288047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8d88b98a-1a95-46c4-927c-8251f5161d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:19.296112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 720/1424] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:35:19.442855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b3440-234a-47c3-8bb1-ac54f5c92852 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:19.457143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b3440-234a-47c3-8bb1-ac54f5c92852 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:19.462383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453bd193-b220-4938-9140-ca873d4162dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:19.463010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bd193-b220-4938-9140-ca873d4162dd service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:19.463324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bd193-b220-4938-9140-ca873d4162dd service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:19.463601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453bd193-b220-4938-9140-ca873d4162dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:19.472272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bd193-b220-4938-9140-ca873d4162dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:19.473851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453bd193-b220-4938-9140-ca873d4162dd service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:19.483981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 705/1425] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:20.324696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de841f60-533f-42a8-9180-2924941f0209 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:20.325254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de841f60-533f-42a8-9180-2924941f0209 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:20.325360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de841f60-533f-42a8-9180-2924941f0209 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:20.327314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 721/1426] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:20.332086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68591971-6d84-4e38-b3b1-1810cc72b191 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:20.332512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68591971-6d84-4e38-b3b1-1810cc72b191 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:20.332651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68591971-6d84-4e38-b3b1-1810cc72b191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:20.350331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-68591971-6d84-4e38-b3b1-1810cc72b191 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:20.351578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-68591971-6d84-4e38-b3b1-1810cc72b191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:20.359406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 706/1427] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:20.509846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de841f60-533f-42a8-9180-2924941f0209 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:20.524038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de841f60-533f-42a8-9180-2924941f0209 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:20.529665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c0d78f-8eec-43cd-9a93-161fb03c7689 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:20.530298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d78f-8eec-43cd-9a93-161fb03c7689 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:20.530582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d78f-8eec-43cd-9a93-161fb03c7689 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:20.530887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c0d78f-8eec-43cd-9a93-161fb03c7689 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:20.538869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0d78f-8eec-43cd-9a93-161fb03c7689 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:20.540470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c0d78f-8eec-43cd-9a93-161fb03c7689 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:20.550215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 722/1428] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:21.347645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea25ff6-0d90-47ac-aea9-56060dd6a757 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:21.348171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea25ff6-0d90-47ac-aea9-56060dd6a757 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:21.348263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea25ff6-0d90-47ac-aea9-56060dd6a757 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:21.349602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 707/1429] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:21.354400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4888b-609b-44e0-af1d-dff058a8572a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:21.354813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4888b-609b-44e0-af1d-dff058a8572a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:21.354963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4888b-609b-44e0-af1d-dff058a8572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:21.372564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-73a4888b-609b-44e0-af1d-dff058a8572a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:21.373752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-73a4888b-609b-44e0-af1d-dff058a8572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:21.381440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 723/1430] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:21.530746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea25ff6-0d90-47ac-aea9-56060dd6a757 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:21.544941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea25ff6-0d90-47ac-aea9-56060dd6a757 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:21.550322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd8b86b-8fba-40cb-b366-43598cda278e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:21.550956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8b86b-8fba-40cb-b366-43598cda278e service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:21.551342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8b86b-8fba-40cb-b366-43598cda278e service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:21.551625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd8b86b-8fba-40cb-b366-43598cda278e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:21.559572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8b86b-8fba-40cb-b366-43598cda278e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:21.561166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd8b86b-8fba-40cb-b366-43598cda278e service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:21.570367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 708/1431] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:22.365506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d122-3a21-47df-9d5f-e3c61db0ef2c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:22.366065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d122-3a21-47df-9d5f-e3c61db0ef2c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:22.366065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4d122-3a21-47df-9d5f-e3c61db0ef2c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:22.367530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 724/1432] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:22.372459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed871c3-0381-488d-b834-30b5aeacf5fc None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:22.372893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed871c3-0381-488d-b834-30b5aeacf5fc None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:22.373053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed871c3-0381-488d-b834-30b5aeacf5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:22.391328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-aed871c3-0381-488d-b834-30b5aeacf5fc None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:22.392511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-aed871c3-0381-488d-b834-30b5aeacf5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:22.400236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 709/1433] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:22.550595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d122-3a21-47df-9d5f-e3c61db0ef2c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:22.564840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4d122-3a21-47df-9d5f-e3c61db0ef2c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:22.569996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed0dab-ad95-40f4-9a06-a1bd72bfc741 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:22.570734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed0dab-ad95-40f4-9a06-a1bd72bfc741 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:22.571020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed0dab-ad95-40f4-9a06-a1bd72bfc741 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:22.571332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed0dab-ad95-40f4-9a06-a1bd72bfc741 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:22.579328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed0dab-ad95-40f4-9a06-a1bd72bfc741 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:22.580935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed0dab-ad95-40f4-9a06-a1bd72bfc741 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:22.590093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 725/1434] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:23.385492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf456a5c-6d38-4214-ac36-4fa10bcab0a0 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:23.386059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf456a5c-6d38-4214-ac36-4fa10bcab0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:23.386059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf456a5c-6d38-4214-ac36-4fa10bcab0a0 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:23.387462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 710/1435] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:23.392918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7a96-e7ad-4eef-b95f-982f2e37388f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:23.393344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7a96-e7ad-4eef-b95f-982f2e37388f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:23.393487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab7a96-e7ad-4eef-b95f-982f2e37388f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:23.411459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-aaab7a96-e7ad-4eef-b95f-982f2e37388f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:23.412658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-aaab7a96-e7ad-4eef-b95f-982f2e37388f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:23.420356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 726/1436] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:23.572338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf456a5c-6d38-4214-ac36-4fa10bcab0a0 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:23.586467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf456a5c-6d38-4214-ac36-4fa10bcab0a0 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:23.591826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e945d26-604d-48dc-ac64-e87893f0f63a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:23.592459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945d26-604d-48dc-ac64-e87893f0f63a service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:23.592740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945d26-604d-48dc-ac64-e87893f0f63a service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:23.593013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e945d26-604d-48dc-ac64-e87893f0f63a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:23.600980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e945d26-604d-48dc-ac64-e87893f0f63a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:23.602558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e945d26-604d-48dc-ac64-e87893f0f63a service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:23.612205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 711/1437] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:24.409380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95bcc-81e8-48d6-b509-e6bd5483e8b2 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:24.409996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95bcc-81e8-48d6-b509-e6bd5483e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:24.409996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95bcc-81e8-48d6-b509-e6bd5483e8b2 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:24.411343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 727/1438] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:24.420888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17d805-25cf-43e8-9a09-c3714c4338af None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:24.421324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17d805-25cf-43e8-9a09-c3714c4338af None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:24.421531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a17d805-25cf-43e8-9a09-c3714c4338af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:24.445429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6a17d805-25cf-43e8-9a09-c3714c4338af None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:24.446564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6a17d805-25cf-43e8-9a09-c3714c4338af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:24.454343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 712/1439] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:35:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:24.611203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95bcc-81e8-48d6-b509-e6bd5483e8b2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:24.625646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95bcc-81e8-48d6-b509-e6bd5483e8b2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:24.630888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a7e3f8-ee38-4c48-8177-8e4f27e3aeec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:24.631564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7e3f8-ee38-4c48-8177-8e4f27e3aeec service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:24.631847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7e3f8-ee38-4c48-8177-8e4f27e3aeec service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:24.632123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a7e3f8-ee38-4c48-8177-8e4f27e3aeec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:24.640101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7e3f8-ee38-4c48-8177-8e4f27e3aeec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:24.641666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a7e3f8-ee38-4c48-8177-8e4f27e3aeec service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:24.650741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 728/1440] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:26.571671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dede6b-c150-4e69-8280-8610497b4235 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:26.572250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dede6b-c150-4e69-8280-8610497b4235 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:26.572250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dede6b-c150-4e69-8280-8610497b4235 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:26.573612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 713/1441] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:26.579349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654cb35-e0f0-4664-9de5-9471ea1db702 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:26.579803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654cb35-e0f0-4664-9de5-9471ea1db702 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:26.579929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654cb35-e0f0-4664-9de5-9471ea1db702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:26.597795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e654cb35-e0f0-4664-9de5-9471ea1db702 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:26.599008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e654cb35-e0f0-4664-9de5-9471ea1db702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:26.606697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 729/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:26.644253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dede6b-c150-4e69-8280-8610497b4235 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:26.655517 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dede6b-c150-4e69-8280-8610497b4235 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:26.658908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb5d37a-d7a1-4448-8d8b-3cbf451d3017 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:26.659570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d37a-d7a1-4448-8d8b-3cbf451d3017 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:26.659849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d37a-d7a1-4448-8d8b-3cbf451d3017 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:26.660118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb5d37a-d7a1-4448-8d8b-3cbf451d3017 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:26.668625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5d37a-d7a1-4448-8d8b-3cbf451d3017 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:26.670050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb5d37a-d7a1-4448-8d8b-3cbf451d3017 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:26.679289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 714/1443] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:28.908719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d289-00bf-41a0-9632-65bdfff71be9 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:28.909266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d289-00bf-41a0-9632-65bdfff71be9 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:28.909266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53d289-00bf-41a0-9632-65bdfff71be9 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:28.910670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 730/1444] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:28.915869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10e592-5f32-4d0c-bffa-afe7af9623cb None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:28.916308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10e592-5f32-4d0c-bffa-afe7af9623cb None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:28.916417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10e592-5f32-4d0c-bffa-afe7af9623cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:28.934279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2a10e592-5f32-4d0c-bffa-afe7af9623cb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:28.935486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2a10e592-5f32-4d0c-bffa-afe7af9623cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:28.943707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 715/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:28.981409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53d289-00bf-41a0-9632-65bdfff71be9 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:28.992407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53d289-00bf-41a0-9632-65bdfff71be9 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:28.995861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbe280-14c8-4627-b9ce-f31d69133d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:28.996492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbe280-14c8-4627-b9ce-f31d69133d8f service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:28.996773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbe280-14c8-4627-b9ce-f31d69133d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:28.997049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbe280-14c8-4627-b9ce-f31d69133d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:29.006277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbe280-14c8-4627-b9ce-f31d69133d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:29.007736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbe280-14c8-4627-b9ce-f31d69133d8f service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:29.017064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 731/1446] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:31.347413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86509f-644d-4bb2-a38b-e88ba6cd973a None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:31.347899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86509f-644d-4bb2-a38b-e88ba6cd973a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:31.347997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86509f-644d-4bb2-a38b-e88ba6cd973a None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:31.349358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 716/1447] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:31.354547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc6581-5a5d-4fbd-91a0-751c010e7853 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:31.354986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc6581-5a5d-4fbd-91a0-751c010e7853 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:31.355130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc6581-5a5d-4fbd-91a0-751c010e7853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:31.373130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-24dc6581-5a5d-4fbd-91a0-751c010e7853 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:31.374323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-24dc6581-5a5d-4fbd-91a0-751c010e7853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:31.382051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 732/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:31.421199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86509f-644d-4bb2-a38b-e88ba6cd973a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:31.431665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86509f-644d-4bb2-a38b-e88ba6cd973a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:31.435109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d313b264-da39-425e-8703-1ed18c7c64d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:31.435812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313b264-da39-425e-8703-1ed18c7c64d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:31.436079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313b264-da39-425e-8703-1ed18c7c64d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:31.436354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d313b264-da39-425e-8703-1ed18c7c64d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:31.444992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313b264-da39-425e-8703-1ed18c7c64d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:31.446410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d313b264-da39-425e-8703-1ed18c7c64d2 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:31.455638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 717/1449] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:33.594547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452268c-8982-4c30-a7f0-238cb576ec01 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:33.595091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452268c-8982-4c30-a7f0-238cb576ec01 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:33.595091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4452268c-8982-4c30-a7f0-238cb576ec01 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:33.596568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 733/1450] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:33.601752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74158-476e-4ce3-969f-659c000a9e25 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:33.602176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74158-476e-4ce3-969f-659c000a9e25 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:33.602291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b74158-476e-4ce3-969f-659c000a9e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:33.619700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-67b74158-476e-4ce3-969f-659c000a9e25 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:33.620875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-67b74158-476e-4ce3-969f-659c000a9e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:33.629229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 718/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:33.667467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452268c-8982-4c30-a7f0-238cb576ec01 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:33.678258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4452268c-8982-4c30-a7f0-238cb576ec01 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:33.681843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011f0d84-43fb-4db4-ac09-c9546aedd720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:33.682508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f0d84-43fb-4db4-ac09-c9546aedd720 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:33.682796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f0d84-43fb-4db4-ac09-c9546aedd720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:33.683076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011f0d84-43fb-4db4-ac09-c9546aedd720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:33.692537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f0d84-43fb-4db4-ac09-c9546aedd720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:33.693957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011f0d84-43fb-4db4-ac09-c9546aedd720 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:33.703929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 734/1452] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:35.840919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c27066-fe51-42b5-b435-e22b2ead4405 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:35.841498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c27066-fe51-42b5-b435-e22b2ead4405 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:35.841498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c27066-fe51-42b5-b435-e22b2ead4405 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:35.843372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 719/1453] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:35.848493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f31cf5-3072-48aa-87dc-0df673b7cf86 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:35.848936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f31cf5-3072-48aa-87dc-0df673b7cf86 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:35.849043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f31cf5-3072-48aa-87dc-0df673b7cf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:35.866985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-75f31cf5-3072-48aa-87dc-0df673b7cf86 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:35.868203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-75f31cf5-3072-48aa-87dc-0df673b7cf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:35.876173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 735/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:35.916476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c27066-fe51-42b5-b435-e22b2ead4405 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:35.918573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-38c27066-fe51-42b5-b435-e22b2ead4405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:35.934577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c27066-fe51-42b5-b435-e22b2ead4405 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:35.938110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c635766-07ab-484f-9d5f-97636f4dbea3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:35.938757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635766-07ab-484f-9d5f-97636f4dbea3 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:35.939042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635766-07ab-484f-9d5f-97636f4dbea3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:35.939358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c635766-07ab-484f-9d5f-97636f4dbea3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:35.948507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635766-07ab-484f-9d5f-97636f4dbea3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:35.949932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c635766-07ab-484f-9d5f-97636f4dbea3 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:35.964648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 720/1455] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:38.090701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae38e6-f600-48e1-8080-6ecd105d247b None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:38.091302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae38e6-f600-48e1-8080-6ecd105d247b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:38.091302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae38e6-f600-48e1-8080-6ecd105d247b None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:38.093251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 736/1456] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:38.098368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a3ffc-2e7e-40cd-b65a-ba90089159ce None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:38.098772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a3ffc-2e7e-40cd-b65a-ba90089159ce None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:38.098924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a3ffc-2e7e-40cd-b65a-ba90089159ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:38.116237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-966a3ffc-2e7e-40cd-b65a-ba90089159ce None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:38.117418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-966a3ffc-2e7e-40cd-b65a-ba90089159ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:38.125636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 721/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:38.163451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae38e6-f600-48e1-8080-6ecd105d247b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:38.174152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae38e6-f600-48e1-8080-6ecd105d247b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:38.177923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5fcba5-d0f6-4b83-a66f-13b8c981aaff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:38.178573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fcba5-d0f6-4b83-a66f-13b8c981aaff service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:38.178850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fcba5-d0f6-4b83-a66f-13b8c981aaff service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:38.179128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fcba5-d0f6-4b83-a66f-13b8c981aaff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:38.188725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fcba5-d0f6-4b83-a66f-13b8c981aaff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:38.190150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5fcba5-d0f6-4b83-a66f-13b8c981aaff service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:38.199472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 737/1458] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:40.280942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d6f10-166a-4d99-9dfc-cc50e608ec01 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:40.281468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d6f10-166a-4d99-9dfc-cc50e608ec01 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:40.281586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d6f10-166a-4d99-9dfc-cc50e608ec01 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:40.282883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 722/1459] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:40.288170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa860-fda3-48cc-a1af-16dafeaa48b5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:40.288618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa860-fda3-48cc-a1af-16dafeaa48b5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:40.288741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0aa860-fda3-48cc-a1af-16dafeaa48b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:40.306512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8d0aa860-fda3-48cc-a1af-16dafeaa48b5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:40.307727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8d0aa860-fda3-48cc-a1af-16dafeaa48b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:40.316049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 738/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:40.352826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d6f10-166a-4d99-9dfc-cc50e608ec01 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:40.364133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d6f10-166a-4d99-9dfc-cc50e608ec01 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:40.367645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda627b9-d327-4835-9bcd-ed2a75d63c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:40.368278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda627b9-d327-4835-9bcd-ed2a75d63c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:40.368589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda627b9-d327-4835-9bcd-ed2a75d63c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:40.368866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda627b9-d327-4835-9bcd-ed2a75d63c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:40.377554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda627b9-d327-4835-9bcd-ed2a75d63c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:40.379008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda627b9-d327-4835-9bcd-ed2a75d63c7a service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:40.388290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 723/1461] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:42.445534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e60f5b-ac26-435a-910e-cef3a7949639 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:42.446055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e60f5b-ac26-435a-910e-cef3a7949639 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:42.446055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e60f5b-ac26-435a-910e-cef3a7949639 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:42.447380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 739/1462] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:42.453251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e35e-6b30-47eb-b793-b90dcbc7ff9f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:42.453666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e35e-6b30-47eb-b793-b90dcbc7ff9f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:42.453827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768e35e-6b30-47eb-b793-b90dcbc7ff9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:42.459826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=79 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:42.473779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e768e35e-6b30-47eb-b793-b90dcbc7ff9f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:42.474986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e768e35e-6b30-47eb-b793-b90dcbc7ff9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:42.482930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 724/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:42.519707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e60f5b-ac26-435a-910e-cef3a7949639 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:42.525000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=73 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:42.532405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e60f5b-ac26-435a-910e-cef3a7949639 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:42.535901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780658e-632b-418c-b888-53e2decf4cdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:42.536546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780658e-632b-418c-b888-53e2decf4cdd service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:42.536833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780658e-632b-418c-b888-53e2decf4cdd service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:42.537111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780658e-632b-418c-b888-53e2decf4cdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:42.546009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780658e-632b-418c-b888-53e2decf4cdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:42.547503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780658e-632b-418c-b888-53e2decf4cdd service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:42.556798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 740/1464] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:44.579330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ed923-fd4f-4fb6-9e7d-f833b820e38e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:44.579885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ed923-fd4f-4fb6-9e7d-f833b820e38e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:44.579885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ed923-fd4f-4fb6-9e7d-f833b820e38e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:44.581258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 725/1465] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:44.586545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62ac6-477b-4e96-bb57-7e1e1b3dbbba None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:44.586963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62ac6-477b-4e96-bb57-7e1e1b3dbbba None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:44.587145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a62ac6-477b-4e96-bb57-7e1e1b3dbbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:44.605132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-56a62ac6-477b-4e96-bb57-7e1e1b3dbbba None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:44.606386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-56a62ac6-477b-4e96-bb57-7e1e1b3dbbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:44.614159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 741/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:35:44.650342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ed923-fd4f-4fb6-9e7d-f833b820e38e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:44.662149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ed923-fd4f-4fb6-9e7d-f833b820e38e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:44.665656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ace5431-9d8e-41e1-82c2-e88fad36dea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:44.666288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace5431-9d8e-41e1-82c2-e88fad36dea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:44.666568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace5431-9d8e-41e1-82c2-e88fad36dea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:44.666843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ace5431-9d8e-41e1-82c2-e88fad36dea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:44.675566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace5431-9d8e-41e1-82c2-e88fad36dea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:44.676988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ace5431-9d8e-41e1-82c2-e88fad36dea8 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:44.686103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 726/1467] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:46.728696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a15e8-4866-47ac-8349-b260669c8be3 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:46.729255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a15e8-4866-47ac-8349-b260669c8be3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:46.729255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843a15e8-4866-47ac-8349-b260669c8be3 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:46.730691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 742/1468] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:35:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:35:46.736359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b00167-3270-4fda-8414-6d83065ea324 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:46.736800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b00167-3270-4fda-8414-6d83065ea324 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:46.736913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b00167-3270-4fda-8414-6d83065ea324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:46.754574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-27b00167-3270-4fda-8414-6d83065ea324 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:35:46.755776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-27b00167-3270-4fda-8414-6d83065ea324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:35:46.763366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 727/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:35:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:35:46.799946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a15e8-4866-47ac-8349-b260669c8be3 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:35:46.810442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843a15e8-4866-47ac-8349-b260669c8be3 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:35:46.814598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d77a3b-8209-4737-a889-d59a6e37f3ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:35:46.815253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77a3b-8209-4737-a889-d59a6e37f3ae service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:35:46.815541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77a3b-8209-4737-a889-d59a6e37f3ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:35:46.815819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d77a3b-8209-4737-a889-d59a6e37f3ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:35:46.824565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d77a3b-8209-4737-a889-d59a6e37f3ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:35:46.825986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d77a3b-8209-4737-a889-d59a6e37f3ae service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:35:46.835413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 743/1470] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:35:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:35:56.750363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=11 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:35:56.831136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=14 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:36:06.563756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8416cd8-ab21-4cad-a75d-468e70daab5b None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:06.564324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8416cd8-ab21-4cad-a75d-468e70daab5b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:06.564324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8416cd8-ab21-4cad-a75d-468e70daab5b None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:06.565749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 728/1471] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:06.571115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797ced-b6d2-4481-b42e-057c7fbca16c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:06.571602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797ced-b6d2-4481-b42e-057c7fbca16c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:06.571709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88797ced-b6d2-4481-b42e-057c7fbca16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:06.588958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-88797ced-b6d2-4481-b42e-057c7fbca16c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:06.590130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-88797ced-b6d2-4481-b42e-057c7fbca16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:06.598256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 744/1472] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:06.645363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d8b22-1266-4c1a-87e8-c7ccefcde003 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:06.645653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d8b22-1266-4c1a-87e8-c7ccefcde003 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:06.645936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9d8b22-1266-4c1a-87e8-c7ccefcde003 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:06.647336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 729/1473] 10.222.0.43 () {54 vars in 962 bytes} [Fri Jun 24 08:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:06.652031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e39279-3e87-410e-aa7f-557c2e202544 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:06.652320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e39279-3e87-410e-aa7f-557c2e202544 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:06.652600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e39279-3e87-410e-aa7f-557c2e202544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:06.670685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-39e39279-3e87-410e-aa7f-557c2e202544 None None] MFA Rules not processed for user `ffea41aca5bd4fd49e2c5f74ac317814`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:06.672512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-39e39279-3e87-410e-aa7f-557c2e202544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:06.678767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 745/1474] 10.222.0.43 () {58 vars in 1059 bytes} [Fri Jun 24 08:36:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:06.685816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d8b22-1266-4c1a-87e8-c7ccefcde003 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:06.696156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9d8b22-1266-4c1a-87e8-c7ccefcde003 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:06.699614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1da3739-51bf-4cfe-a672-b938de83d842 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:06.700230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3739-51bf-4cfe-a672-b938de83d842 service glance] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:06.700510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3739-51bf-4cfe-a672-b938de83d842 service glance] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:06.700787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1da3739-51bf-4cfe-a672-b938de83d842 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:06.709490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3739-51bf-4cfe-a672-b938de83d842 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:06.710919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1da3739-51bf-4cfe-a672-b938de83d842 service glance] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:06.720580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 730/1475] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:08.806640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935a510-7bb6-43ad-bd66-54dffb1d5c45 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:08.807188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935a510-7bb6-43ad-bd66-54dffb1d5c45 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:08.807314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7935a510-7bb6-43ad-bd66-54dffb1d5c45 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:08.808667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 746/1476] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:08.813770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff463866-9846-48c4-95c3-d6154649f471 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:08.814255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff463866-9846-48c4-95c3-d6154649f471 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:08.814370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff463866-9846-48c4-95c3-d6154649f471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:08.832387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ff463866-9846-48c4-95c3-d6154649f471 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:08.833559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ff463866-9846-48c4-95c3-d6154649f471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:08.841123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 731/1477] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:08.864223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935a510-7bb6-43ad-bd66-54dffb1d5c45 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:08.877242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7935a510-7bb6-43ad-bd66-54dffb1d5c45 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:08.881702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:08.882346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:08.882630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:08.882924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:08.883673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:08.885427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:36:08.885427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:36:08.885427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:36:08.886562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e11aef0-1b5c-4d10-ae92-646172719a99 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:08.906856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 747/1478] 10.222.0.43 () {56 vars in 1108 bytes} [Fri Jun 24 08:36:08 2022] GET /identity/v3/projects => generated 2553 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:36:10.937866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a090f-9a77-4e39-91a4-2b55d942aa46 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:10.938451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a090f-9a77-4e39-91a4-2b55d942aa46 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:10.938451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a090f-9a77-4e39-91a4-2b55d942aa46 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:10.939806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 732/1479] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:10.944937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7bdf-50f4-4b5e-bcd9-9c549f37cb86 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:10.945389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7bdf-50f4-4b5e-bcd9-9c549f37cb86 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:10.945495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648c7bdf-50f4-4b5e-bcd9-9c549f37cb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:10.962942 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-648c7bdf-50f4-4b5e-bcd9-9c549f37cb86 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:10.964227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-648c7bdf-50f4-4b5e-bcd9-9c549f37cb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:10.971968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 748/1480] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:11.014496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a090f-9a77-4e39-91a4-2b55d942aa46 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:11.029342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a090f-9a77-4e39-91a4-2b55d942aa46 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:11.034907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8371c7c-1258-45b9-a29f-454a1547bc1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:11.035575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8371c7c-1258-45b9-a29f-454a1547bc1b service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:11.035862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8371c7c-1258-45b9-a29f-454a1547bc1b service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:11.036142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8371c7c-1258-45b9-a29f-454a1547bc1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:11.044090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8371c7c-1258-45b9-a29f-454a1547bc1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:11.045677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8371c7c-1258-45b9-a29f-454a1547bc1b service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:11.055327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 733/1481] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:13.260578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b6f1c-32f0-4b2e-8e80-ce3084130ef4 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:13.261161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b6f1c-32f0-4b2e-8e80-ce3084130ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:13.261161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3b6f1c-32f0-4b2e-8e80-ce3084130ef4 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:13.262498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 749/1482] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:13.267598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d2521-4772-46be-9730-7a8e37f781aa None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:13.268040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d2521-4772-46be-9730-7a8e37f781aa None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:13.268148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d2521-4772-46be-9730-7a8e37f781aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:13.285695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ce0d2521-4772-46be-9730-7a8e37f781aa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:13.286879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ce0d2521-4772-46be-9730-7a8e37f781aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:13.294575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 734/1483] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:13.331571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b6f1c-32f0-4b2e-8e80-ce3084130ef4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:13.342075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3b6f1c-32f0-4b2e-8e80-ce3084130ef4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:13.346010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330feac-0ffb-4b91-8ec8-1851cc1f5c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:13.346636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330feac-0ffb-4b91-8ec8-1851cc1f5c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:13.346946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330feac-0ffb-4b91-8ec8-1851cc1f5c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:13.347242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330feac-0ffb-4b91-8ec8-1851cc1f5c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:13.356127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330feac-0ffb-4b91-8ec8-1851cc1f5c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:13.357543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330feac-0ffb-4b91-8ec8-1851cc1f5c10 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:13.367477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 750/1484] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:15.447055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d02890-2ffa-4240-8dbc-2f507ad1a729 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:15.447631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d02890-2ffa-4240-8dbc-2f507ad1a729 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:15.447631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d02890-2ffa-4240-8dbc-2f507ad1a729 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:15.448989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 735/1485] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:15.454858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa652b8-93ee-4487-bb9c-9785420d5f2f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:15.455303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa652b8-93ee-4487-bb9c-9785420d5f2f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:15.455455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa652b8-93ee-4487-bb9c-9785420d5f2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:15.474019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5aa652b8-93ee-4487-bb9c-9785420d5f2f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:15.475205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5aa652b8-93ee-4487-bb9c-9785420d5f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:15.482941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 751/1486] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:36:15.520193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d02890-2ffa-4240-8dbc-2f507ad1a729 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:15.530986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d02890-2ffa-4240-8dbc-2f507ad1a729 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:15.535062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37a2d4-82f9-4419-9c3e-2b76453ee039 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:15.535726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37a2d4-82f9-4419-9c3e-2b76453ee039 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:15.536013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37a2d4-82f9-4419-9c3e-2b76453ee039 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:15.536293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37a2d4-82f9-4419-9c3e-2b76453ee039 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:15.545011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37a2d4-82f9-4419-9c3e-2b76453ee039 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:15.546445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37a2d4-82f9-4419-9c3e-2b76453ee039 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:15.555727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 736/1487] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:17.672875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8290d1-5c39-4bde-8976-68bc74948d0c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:17.673420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8290d1-5c39-4bde-8976-68bc74948d0c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:17.673420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8290d1-5c39-4bde-8976-68bc74948d0c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:17.674819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 752/1488] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:17.679984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2e33a-c14f-42c8-83f5-019f1d8c4ddf None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:17.680400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2e33a-c14f-42c8-83f5-019f1d8c4ddf None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:17.680594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f2e33a-c14f-42c8-83f5-019f1d8c4ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:17.697995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-77f2e33a-c14f-42c8-83f5-019f1d8c4ddf None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:17.699182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-77f2e33a-c14f-42c8-83f5-019f1d8c4ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:17.707371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 737/1489] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:17.747451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8290d1-5c39-4bde-8976-68bc74948d0c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:17.758022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8290d1-5c39-4bde-8976-68bc74948d0c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:17.761503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01662845-6a1d-49ac-b9e1-28410ad72f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:17.762136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01662845-6a1d-49ac-b9e1-28410ad72f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:17.762421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01662845-6a1d-49ac-b9e1-28410ad72f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:17.762700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01662845-6a1d-49ac-b9e1-28410ad72f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:17.772058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01662845-6a1d-49ac-b9e1-28410ad72f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:17.773480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01662845-6a1d-49ac-b9e1-28410ad72f34 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:17.783915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 753/1490] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:19.975914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f9cfb-d3b6-445c-94db-d7dd4587fcdb None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:19.976483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f9cfb-d3b6-445c-94db-d7dd4587fcdb None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:19.976483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f9cfb-d3b6-445c-94db-d7dd4587fcdb None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:19.977849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 738/1491] 10.222.0.43 () {54 vars in 926 bytes} [Fri Jun 24 08:36:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:36:19.982947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4cac-5507-4fc1-b6ed-c6eadef58ab7 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:19.983378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4cac-5507-4fc1-b6ed-c6eadef58ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:19.983550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c4cac-5507-4fc1-b6ed-c6eadef58ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:20.000829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e53c4cac-5507-4fc1-b6ed-c6eadef58ab7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:36:20.002010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e53c4cac-5507-4fc1-b6ed-c6eadef58ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:36:20.009658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 754/1492] 10.222.0.43 () {58 vars in 1023 bytes} [Fri Jun 24 08:36:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:36:20.048539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f9cfb-d3b6-445c-94db-d7dd4587fcdb None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:36:20.059204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f9cfb-d3b6-445c-94db-d7dd4587fcdb None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:36:20.062757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a093c3-3784-4923-8d81-c5e067318a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:36:20.063426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a093c3-3784-4923-8d81-c5e067318a1a service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:36:20.063712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a093c3-3784-4923-8d81-c5e067318a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:36:20.063993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a093c3-3784-4923-8d81-c5e067318a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:36:20.072712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a093c3-3784-4923-8d81-c5e067318a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:36:20.074134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a093c3-3784-4923-8d81-c5e067318a1a service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:36:20.083983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 739/1493] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:36:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:36:29.995759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=33 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:36:30.079790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=33 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:37:43.631645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.632243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.632243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.649615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:37:43.650786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:43.659148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 755/1494] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:37:43.668190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.668662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.668761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.686809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:37:43.688147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:43.696232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 740/1495] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:37:43.700623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:43.710668 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc38315-e3cb-4e9c-88c0-cbd6e3b826e6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:43.715051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:43.715733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.716015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.716289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.717002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:37:43.717306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:43.718305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:43.718305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:37:43.718305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:37:43.719187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:43.730833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 756/1496] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:37:43.738591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:43.747679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20fc6b-97ee-4417-8f8c-f0bf899b7e2b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:43.752216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:43.752858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.753146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.753426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.754109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:43.755912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:43.755912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:37:43.755912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:37:43.757113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:43.793569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 741/1497] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:37:43 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:37:43.797807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:43.807845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e91936-bae0-4c93-ae85-1f54f83fd216 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:43.862003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:43.862663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.862949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.863248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.863931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:43.864929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:43.864929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:37:43.864929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:37:43.866130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:43.885005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 757/1498] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:37:43 2022] POST /identity/v3/users => generated 290 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:37:43.889084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:43.899722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872229f1-6e07-49f3-a9e9-57cddc684327 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:43.904221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:43.904878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.905166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.905448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.906210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:43.907232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:43.907232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:37:43.907232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:37:43.908751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:43.923118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 742/1499] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:37:43.927271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:43.937340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d13a1-59cf-4a82-88fa-989fee808fcc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:43.941821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:43.942505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.942786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.943062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.943835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:43.944837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:43.944837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:37:43.944837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:37:43.945715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be580d97-02e1-48ae-bad1-cf3cb8841fc4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:43.959230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 758/1500] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:37:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:37:43.963347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:43.973844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe658e-dca3-494b-b684-4f2a7d559921 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:43.978340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:43.979226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:43.979530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:43.979809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] PATH_INFO: `/v3/projects/50c466debf83435086ab12409f2819bc/users/0af0e690290f480489e9a2764665cf58/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:43.995126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c466debf83435086ab12409f2819bc, user_id=0af0e690290f480489e9a2764665cf58, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:43.996191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:43.996191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:37:43.996191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:37:43.998149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67763fd6-9705-4f37-a4b4-d2684c5914e1 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:44.011077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 743/1501] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:37:43 2022] PUT /identity/v3/projects/50c466debf83435086ab12409f2819bc/users/0af0e690290f480489e9a2764665cf58/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:37:44.022691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29eb23-1517-4e41-9f6a-5b9a7c49e74e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.022992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29eb23-1517-4e41-9f6a-5b9a7c49e74e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.023297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa29eb23-1517-4e41-9f6a-5b9a7c49e74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.049087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-fa29eb23-1517-4e41-9f6a-5b9a7c49e74e None None] MFA Rules not processed for user `0af0e690290f480489e9a2764665cf58`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:37:44.050282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fa29eb23-1517-4e41-9f6a-5b9a7c49e74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:44.112575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 759/1502] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:37:44 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:37:44.125651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ca8d-f26c-4cda-b038-f7ae29928ed9 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.125938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ca8d-f26c-4cda-b038-f7ae29928ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.126216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7621ca8d-f26c-4cda-b038-f7ae29928ed9 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.128104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 744/1503] 10.222.0.43 () {54 vars in 963 bytes} [Fri Jun 24 08:37:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:37:44.132710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2276a-c4d7-4198-8e80-2e4bb98f74ec None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.133062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2276a-c4d7-4198-8e80-2e4bb98f74ec None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.133280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2276a-c4d7-4198-8e80-2e4bb98f74ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.151232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e1c2276a-c4d7-4198-8e80-2e4bb98f74ec None None] MFA Rules not processed for user `f3229f7fdd344dda97a47709c93ce2e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:37:44.152382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e1c2276a-c4d7-4198-8e80-2e4bb98f74ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:44.195739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:37:44.203851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 760/1504] 10.222.0.43 () {58 vars in 1060 bytes} [Fri Jun 24 08:37:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:37:44.212146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ca8d-f26c-4cda-b038-f7ae29928ed9 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:44.222405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7621ca8d-f26c-4cda-b038-f7ae29928ed9 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:44.225828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:44.226456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.226734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.227006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.234971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:44.236420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:44.245015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 745/1505] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:37:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:37:44.286853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b495-b58e-471a-9f75-cb991e64302d None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.287141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b495-b58e-471a-9f75-cb991e64302d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.287459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9b495-b58e-471a-9f75-cb991e64302d None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.288741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 761/1506] 10.222.0.43 () {52 vars in 829 bytes} [Fri Jun 24 08:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:37:44.292410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:44.357243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9937c-204b-4e4e-8836-b249e8d61583 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:44.361751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:44.362609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.362908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.363165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] PATH_INFO: `/v3/users/0af0e690290f480489e9a2764665cf58` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.364374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0af0e690290f480489e9a2764665cf58)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:44.365390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:44.365390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:37:44.365390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:37:44.366595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:44.433124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0af0e690290f480489e9a2764665cf58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:44.442605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0af0e690290f480489e9a2764665cf58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:44.460731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0af0e690290f480489e9a2764665cf58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:44.469077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:37:44.473168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0af0e690290f480489e9a2764665cf58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:44.473979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0af0e690290f480489e9a2764665cf58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:44.481564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 746/1507] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:37:44 2022] DELETE /identity/v3/users/0af0e690290f480489e9a2764665cf58 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:37:44.486001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.486811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.487114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.504905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:37:44.506102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:44.570870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 762/1508] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:37:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:37:44.580381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:44.581798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:44.637702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93e5b-9b61-4b6a-9616-4cf632aee5c9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:44.641308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3228f-d053-46af-8651-76a9e17c3241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:44.641951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3228f-d053-46af-8651-76a9e17c3241 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:44.642235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3228f-d053-46af-8651-76a9e17c3241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:44.642542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3228f-d053-46af-8651-76a9e17c3241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:44.651299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3228f-d053-46af-8651-76a9e17c3241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:44.652735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3228f-d053-46af-8651-76a9e17c3241 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:44.662234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 747/1509] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:37:44.992490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:37:44.993952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:37:45.005495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15665134-77fb-4ef5-82de-269abc0aa5c4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:37:45.010162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:37:45.011888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:37:45.012198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:37:45.012473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] PATH_INFO: `/v3/projects/50c466debf83435086ab12409f2819bc` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:37:45.013675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c466debf83435086ab12409f2819bc)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:37:45.014692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:37:45.014692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:37:45.014692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:37:45.016522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:37:45.077512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c466debf83435086ab12409f2819bc', 'request_id': 'req-7be55267-eb40-41f7-a01b-c7c06e0664cd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:45.083659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c466debf83435086ab12409f2819bc', 'request_id': 'req-7be55267-eb40-41f7-a01b-c7c06e0664cd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:45.097180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c466debf83435086ab12409f2819bc', 'request_id': 'req-7be55267-eb40-41f7-a01b-c7c06e0664cd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:45.112993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] The token cache is being invalidate because project 50c466debf83435086ab12409f2819bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:37:45.113285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:45.113958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7be55267-eb40-41f7-a01b-c7c06e0664cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:37:45.115675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 763/1510] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:37:44 2022] DELETE /identity/v3/projects/50c466debf83435086ab12409f2819bc => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:37:54.657827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=29 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:37:55.109374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:39:28.750936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e37584-6af5-4e9a-a90e-5b8fec967221 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:28.751548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e37584-6af5-4e9a-a90e-5b8fec967221 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:28.751548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e37584-6af5-4e9a-a90e-5b8fec967221 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:28.752930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 748/1511] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:28.757754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002286c-8019-47fd-be47-c3cdda7a2c49 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:28.758187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002286c-8019-47fd-be47-c3cdda7a2c49 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:28.758312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002286c-8019-47fd-be47-c3cdda7a2c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:28.781126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e002286c-8019-47fd-be47-c3cdda7a2c49 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:28.782396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e002286c-8019-47fd-be47-c3cdda7a2c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:28.839803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 764/1512] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:28.993308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e37584-6af5-4e9a-a90e-5b8fec967221 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:28.995002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-16e37584-6af5-4e9a-a90e-5b8fec967221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:29.068724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e37584-6af5-4e9a-a90e-5b8fec967221 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:29.074067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea274a4c-2fb0-46c8-936c-ba934039f233 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:29.074707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea274a4c-2fb0-46c8-936c-ba934039f233 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:29.074995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea274a4c-2fb0-46c8-936c-ba934039f233 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:29.075312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea274a4c-2fb0-46c8-936c-ba934039f233 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:29.083876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea274a4c-2fb0-46c8-936c-ba934039f233 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:29.085478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea274a4c-2fb0-46c8-936c-ba934039f233 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:29.095035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 749/1513] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:29.932806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff680-df7c-4467-84f8-aa9094614d25 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:29.933467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff680-df7c-4467-84f8-aa9094614d25 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:29.933467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1ff680-df7c-4467-84f8-aa9094614d25 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:29.934788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 765/1514] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:39:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:29.939572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087232c-21b3-4984-bf60-c7aaf8d5a1bf None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:29.940000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087232c-21b3-4984-bf60-c7aaf8d5a1bf None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:29.940134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3087232c-21b3-4984-bf60-c7aaf8d5a1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:29.959026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3087232c-21b3-4984-bf60-c7aaf8d5a1bf None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:29.960279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3087232c-21b3-4984-bf60-c7aaf8d5a1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:29.968127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 750/1515] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:29.985973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b362f3-eb41-4dd8-941f-2405e95c6e8a None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:29.986273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b362f3-eb41-4dd8-941f-2405e95c6e8a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:29.986560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b362f3-eb41-4dd8-941f-2405e95c6e8a None None] PATH_INFO: `/v3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:29.987889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 766/1516] 10.222.0.46 () {54 vars in 925 bytes} [Fri Jun 24 08:39:29 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:39:29.995007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c71a79-d795-444b-bd1b-2c6b01990401 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:29.995304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c71a79-d795-444b-bd1b-2c6b01990401 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:29.995583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c71a79-d795-444b-bd1b-2c6b01990401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:30.013380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a0c71a79-d795-444b-bd1b-2c6b01990401 None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:30.014546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a0c71a79-d795-444b-bd1b-2c6b01990401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:30.076336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 751/1517] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Jun 24 08:39:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:30.117961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b362f3-eb41-4dd8-941f-2405e95c6e8a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:30.132785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b362f3-eb41-4dd8-941f-2405e95c6e8a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:30.138069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4658c0-f37a-4a9a-8bf9-77ff9f543fc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:30.138704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4658c0-f37a-4a9a-8bf9-77ff9f543fc9 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:30.138986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4658c0-f37a-4a9a-8bf9-77ff9f543fc9 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:30.139286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4658c0-f37a-4a9a-8bf9-77ff9f543fc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:30.147533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4658c0-f37a-4a9a-8bf9-77ff9f543fc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:30.149124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4658c0-f37a-4a9a-8bf9-77ff9f543fc9 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:30.158568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 767/1518] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:39.055302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f0f95-3ec9-4783-bb39-835a9fa90f2e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:39.055853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f0f95-3ec9-4783-bb39-835a9fa90f2e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:39.055853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554f0f95-3ec9-4783-bb39-835a9fa90f2e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:39.057223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 752/1519] 10.222.0.43 () {54 vars in 960 bytes} [Fri Jun 24 08:39:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:39.062300 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58417577-9255-4da7-9ec0-90e45322663d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:39.062730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58417577-9255-4da7-9ec0-90e45322663d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:39.062847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58417577-9255-4da7-9ec0-90e45322663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:39.080456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-58417577-9255-4da7-9ec0-90e45322663d None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:39.081622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58417577-9255-4da7-9ec0-90e45322663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:39.090465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 768/1520] 10.222.0.43 () {58 vars in 1057 bytes} [Fri Jun 24 08:39:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:39.097550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f0f95-3ec9-4783-bb39-835a9fa90f2e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:39.111256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554f0f95-3ec9-4783-bb39-835a9fa90f2e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:39.116040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1746-568d-4ca2-8dbd-664a67dca0b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:39.116661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1746-568d-4ca2-8dbd-664a67dca0b5 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:39.116942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1746-568d-4ca2-8dbd-664a67dca0b5 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:39.117218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1746-568d-4ca2-8dbd-664a67dca0b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:39.125213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1746-568d-4ca2-8dbd-664a67dca0b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:39.126794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1746-568d-4ca2-8dbd-664a67dca0b5 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:39.136106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 753/1521] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:41.370818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0e520-f509-4fb8-8e9b-92073cf9064f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:41.371406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0e520-f509-4fb8-8e9b-92073cf9064f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:41.371406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0e520-f509-4fb8-8e9b-92073cf9064f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:41.372818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 769/1522] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:41.377495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5938df8-9679-43dc-b117-afea6a9d6702 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:41.377922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5938df8-9679-43dc-b117-afea6a9d6702 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:41.378064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5938df8-9679-43dc-b117-afea6a9d6702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:41.398125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c5938df8-9679-43dc-b117-afea6a9d6702 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:41.399346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c5938df8-9679-43dc-b117-afea6a9d6702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:41.407003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 754/1523] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:39:41.560598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0e520-f509-4fb8-8e9b-92073cf9064f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:41.575345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0e520-f509-4fb8-8e9b-92073cf9064f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:41.580675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b47067-38c4-4bc5-ba5f-006a5c8a7d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:41.581313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b47067-38c4-4bc5-ba5f-006a5c8a7d19 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:41.581595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b47067-38c4-4bc5-ba5f-006a5c8a7d19 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:41.581872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b47067-38c4-4bc5-ba5f-006a5c8a7d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:41.589913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b47067-38c4-4bc5-ba5f-006a5c8a7d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:41.591532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b47067-38c4-4bc5-ba5f-006a5c8a7d19 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:41.600776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 770/1524] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:39:42.423363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da1564-8cc0-4cbc-9a30-2e09500a70ae None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:42.423926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da1564-8cc0-4cbc-9a30-2e09500a70ae None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:42.423926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4da1564-8cc0-4cbc-9a30-2e09500a70ae None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:42.425331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 755/1525] 127.0.0.1 () {54 vars in 901 bytes} [Fri Jun 24 08:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:42.430643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b6ed-039b-4062-9334-5cfd4e79f733 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:42.431033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b6ed-039b-4062-9334-5cfd4e79f733 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:42.431134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9b6ed-039b-4062-9334-5cfd4e79f733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:42.449224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-05e9b6ed-039b-4062-9334-5cfd4e79f733 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:42.450442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-05e9b6ed-039b-4062-9334-5cfd4e79f733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:42.458324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 771/1526] 127.0.0.1 () {58 vars in 998 bytes} [Fri Jun 24 08:39:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:42.607729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da1564-8cc0-4cbc-9a30-2e09500a70ae None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:42.622583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4da1564-8cc0-4cbc-9a30-2e09500a70ae None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:42.627930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70694d85-8ab7-413a-9f61-2b83e8f8cf7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:42.628563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694d85-8ab7-413a-9f61-2b83e8f8cf7c service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:42.628846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694d85-8ab7-413a-9f61-2b83e8f8cf7c service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:42.629121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70694d85-8ab7-413a-9f61-2b83e8f8cf7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:42.637161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70694d85-8ab7-413a-9f61-2b83e8f8cf7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:42.638729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70694d85-8ab7-413a-9f61-2b83e8f8cf7c service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:42.647884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 756/1527] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:47.328480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36286503-8889-4416-99bb-372117366bcb None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:47.329072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36286503-8889-4416-99bb-372117366bcb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:47.329072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36286503-8889-4416-99bb-372117366bcb None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:47.330449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 772/1528] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:39:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:47.336634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d71ca-c45a-4fb6-a7a8-f6fb7b300e16 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:47.337073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d71ca-c45a-4fb6-a7a8-f6fb7b300e16 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:47.337189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7d71ca-c45a-4fb6-a7a8-f6fb7b300e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:47.354482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fb7d71ca-c45a-4fb6-a7a8-f6fb7b300e16 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:47.355693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fb7d71ca-c45a-4fb6-a7a8-f6fb7b300e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:47.363347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 757/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:39:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:47.400449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36286503-8889-4416-99bb-372117366bcb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:47.402124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-36286503-8889-4416-99bb-372117366bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:47.459685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36286503-8889-4416-99bb-372117366bcb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:47.463094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dabd2-ce72-4aa1-8017-5b2409f9127e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:47.463763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dabd2-ce72-4aa1-8017-5b2409f9127e service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:47.464050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dabd2-ce72-4aa1-8017-5b2409f9127e service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:47.464331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dabd2-ce72-4aa1-8017-5b2409f9127e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:47.473051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dabd2-ce72-4aa1-8017-5b2409f9127e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:47.475012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dabd2-ce72-4aa1-8017-5b2409f9127e service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:47.484285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 773/1530] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:49.509329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a1d5c-a5e9-48b8-b480-b95519717660 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:49.509857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a1d5c-a5e9-48b8-b480-b95519717660 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:49.509857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a1d5c-a5e9-48b8-b480-b95519717660 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:49.511195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 758/1531] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:39:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:49.516256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067c68-0b66-455e-ad22-77d58d06f783 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:49.516707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067c68-0b66-455e-ad22-77d58d06f783 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:49.516816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067c68-0b66-455e-ad22-77d58d06f783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:49.534921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4b067c68-0b66-455e-ad22-77d58d06f783 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:49.536141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4b067c68-0b66-455e-ad22-77d58d06f783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:49.544225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 774/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:39:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:49.589346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a1d5c-a5e9-48b8-b480-b95519717660 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:49.600033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a1d5c-a5e9-48b8-b480-b95519717660 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:49.603521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776037ce-195d-4cb1-8ce7-6edfd01fed62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:49.604154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776037ce-195d-4cb1-8ce7-6edfd01fed62 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:49.604438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776037ce-195d-4cb1-8ce7-6edfd01fed62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:49.604708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776037ce-195d-4cb1-8ce7-6edfd01fed62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:49.613327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776037ce-195d-4cb1-8ce7-6edfd01fed62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:49.615250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776037ce-195d-4cb1-8ce7-6edfd01fed62 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:49.624539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 759/1533] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:53.093350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb81745-f601-431a-bfad-d3a4c4bd8499 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:53.093886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb81745-f601-431a-bfad-d3a4c4bd8499 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:53.093886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb81745-f601-431a-bfad-d3a4c4bd8499 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:53.095290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 775/1534] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:39:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:53.100459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81079f-a0ff-40da-adc6-5d0f0a28bbd9 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:53.100910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81079f-a0ff-40da-adc6-5d0f0a28bbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:53.101055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd81079f-a0ff-40da-adc6-5d0f0a28bbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:53.118609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-cd81079f-a0ff-40da-adc6-5d0f0a28bbd9 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:53.119818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cd81079f-a0ff-40da-adc6-5d0f0a28bbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:53.127431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 760/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:39:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:53.433898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb81745-f601-431a-bfad-d3a4c4bd8499 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:53.435576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-aeb81745-f601-431a-bfad-d3a4c4bd8499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:53.491336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb81745-f601-431a-bfad-d3a4c4bd8499 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:53.494840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7ffc7-ac2e-4f7f-8a11-30d7ddcbe2ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:53.495515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7ffc7-ac2e-4f7f-8a11-30d7ddcbe2ff service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:53.495805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7ffc7-ac2e-4f7f-8a11-30d7ddcbe2ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:53.496087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7ffc7-ac2e-4f7f-8a11-30d7ddcbe2ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:53.504972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7ffc7-ac2e-4f7f-8a11-30d7ddcbe2ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:53.506938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7ffc7-ac2e-4f7f-8a11-30d7ddcbe2ff service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:53.511898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:39:53.516902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 776/1536] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:39:55.554261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e354f-b087-46ce-8bec-1142f2c56dd6 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:55.554881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e354f-b087-46ce-8bec-1142f2c56dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:55.554881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e354f-b087-46ce-8bec-1142f2c56dd6 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:55.556221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 761/1537] 127.0.0.1 () {54 vars in 916 bytes} [Fri Jun 24 08:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:39:55.561292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9623a-22af-4d14-820f-c40452d3f76b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:55.561720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9623a-22af-4d14-820f-c40452d3f76b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:55.561839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d9623a-22af-4d14-820f-c40452d3f76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:55.579415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d2d9623a-22af-4d14-820f-c40452d3f76b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:39:55.580598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d2d9623a-22af-4d14-820f-c40452d3f76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:55.588811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 777/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Fri Jun 24 08:39:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:39:55.887653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e354f-b087-46ce-8bec-1142f2c56dd6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:39:55.889366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7e1e354f-b087-46ce-8bec-1142f2c56dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:39:55.896836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=94 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:39:55.903226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e354f-b087-46ce-8bec-1142f2c56dd6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:39:55.906763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d28803-d5f7-4b41-a2bc-62add41f3fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:39:55.907434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28803-d5f7-4b41-a2bc-62add41f3fbb service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:39:55.907718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28803-d5f7-4b41-a2bc-62add41f3fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:39:55.907994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d28803-d5f7-4b41-a2bc-62add41f3fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:39:55.917189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d28803-d5f7-4b41-a2bc-62add41f3fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:39:55.918612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d28803-d5f7-4b41-a2bc-62add41f3fbb service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:39:55.930012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 762/1539] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:39:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:40:05.574261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:40:05.924325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:27.176043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.176746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.176746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.197422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.204458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.217573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 778/1540] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.229209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.229824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.229824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.261355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.262777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.279344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 763/1541] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:27.283988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.303704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43298b2-6bbc-44c1-8821-66363c243efe None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.309959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.310706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.310998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.311300 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.312042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:27.312400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.314021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d9076c-1c7c-4d0e-bf55-97272d504497 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.336709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 779/1542] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:27.345229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.361382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd3888-46c9-4ee0-92cb-b8900bb99cbc None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.368672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.369872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.370184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.370485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.371236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.373244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.390470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030d85b-d5cf-4b63-be53-44677b539ae8 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.390833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030d85b-d5cf-4b63-be53-44677b539ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.391129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9030d85b-d5cf-4b63-be53-44677b539ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.432636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9030d85b-d5cf-4b63-be53-44677b539ae8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.433892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9030d85b-d5cf-4b63-be53-44677b539ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.437561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 764/1543] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/projects => generated 391 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:27.442348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.551295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09aa7152-5dee-4ef3-9b9b-df2a1b04c42e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.551671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 780/1544] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.563872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.564266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.564454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.568026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.569057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.569356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.569642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.570368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.571449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:41:27.571449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:41:27.571449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:41:27.572681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71015631-2cc4-415c-a827-caf63ce4f877 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.595979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.597323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.617426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 765/1545] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:27.618755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d4b0-15e2-46d2-a3c8-54025e37cbd6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.619041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d4b0-15e2-46d2-a3c8-54025e37cbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.628082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d27d4b0-15e2-46d2-a3c8-54025e37cbd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.632232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 781/1546] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:27.633142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.652930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3b56e2-23a3-4b02-805f-45e7ad365fbd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.662385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-0d27d4b0-15e2-46d2-a3c8-54025e37cbd6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.665175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.665863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.666152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.666440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.667166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.668604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0d27d4b0-15e2-46d2-a3c8-54025e37cbd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.670318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.680697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 766/1547] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.682101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fc051-9e55-4d75-b857-7daf1e824d90 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.682391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fc051-9e55-4d75-b857-7daf1e824d90 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.682724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1fc051-9e55-4d75-b857-7daf1e824d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.683871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 782/1548] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:27.684593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.699984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8ea609-cfd2-4dc3-b3d8-bf50525ee1b7 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.706644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.707396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.707640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3f1fc051-9e55-4d75-b857-7daf1e824d90 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.707771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.707967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.709397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:27.709595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3f1fc051-9e55-4d75-b857-7daf1e824d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.712085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.712085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26e1835-0337-4edf-bb47-0a09eabc12cf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.718137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 767/1549] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.720748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce61a-a38d-4a23-85f4-ad430f14c7c6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.721052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce61a-a38d-4a23-85f4-ad430f14c7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.721337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabce61a-a38d-4a23-85f4-ad430f14c7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.726132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 783/1550] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:27.727586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.727881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.728164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.746435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-dabce61a-a38d-4a23-85f4-ad430f14c7c6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.746642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.747665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dabce61a-a38d-4a23-85f4-ad430f14c7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.747800 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.756101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 784/1552] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.756241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 768/1552] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.757534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.759296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81386f1-45eb-474c-a877-a0098090434e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.759527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81386f1-45eb-474c-a877-a0098090434e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.759809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81386f1-45eb-474c-a877-a0098090434e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.770469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3765a-136b-475b-8722-3125a3eae76f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.775647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7385c5a7-8bca-479d-965f-27c028609bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.776436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385c5a7-8bca-479d-965f-27c028609bae admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.776726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385c5a7-8bca-479d-965f-27c028609bae admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.777008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7385c5a7-8bca-479d-965f-27c028609bae admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.777734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385c5a7-8bca-479d-965f-27c028609bae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.779386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7385c5a7-8bca-479d-965f-27c028609bae admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.780890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a81386f1-45eb-474c-a877-a0098090434e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.782121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a81386f1-45eb-474c-a877-a0098090434e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.789386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 785/1553] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:27.793846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.793978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 769/1554] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.794081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.794081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.794251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fefb4-16e8-4a82-8184-042709dadc98 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.794251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fefb4-16e8-4a82-8184-042709dadc98 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.794251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fefb4-16e8-4a82-8184-042709dadc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.809555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.810642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-8a6fefb4-16e8-4a82-8184-042709dadc98 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.810774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.811815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8a6fefb4-16e8-4a82-8184-042709dadc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.819045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 786/1555] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.819830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.820254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 770/1556] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.821568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72b46-20ad-43e9-be49-0f1183269962 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.821858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72b46-20ad-43e9-be49-0f1183269962 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.822137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe72b46-20ad-43e9-be49-0f1183269962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.832588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5273e9-3959-44ae-8de8-934730edfd62 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.837699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.838382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.838659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.838934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.839657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.840664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:41:27.840664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:41:27.840664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:41:27.841842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c855e-ef21-4069-b1a5-7826112bcc54 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.844632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-afe72b46-20ad-43e9-be49-0f1183269962 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.845808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-afe72b46-20ad-43e9-be49-0f1183269962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.853791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 771/1557] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.854801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.855089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.855396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.870145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 787/1558] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:27.872643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.872939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.873346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.874840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.876057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.892022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:27.893205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:27.934907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 772/1559] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.935674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.949533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e778ac-cb4d-4c8e-8d87-8e40f25e4c69 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.953965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faedebd-8cdc-4a45-a338-cf316f37c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.954466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 788/1560] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:27.954844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faedebd-8cdc-4a45-a338-cf316f37c155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.955117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.955250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faedebd-8cdc-4a45-a338-cf316f37c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.955427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faedebd-8cdc-4a45-a338-cf316f37c155 admin admin] PATH_INFO: `/v3/projects/edde4704253a4439bd7ca50b26e24e40/users/9abdce8004e24e81bd88bee757d8ec93/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.963020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:27.967115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632cd27-86ea-4015-8a4c-1b7b37c14d3f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:27.972855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faedebd-8cdc-4a45-a338-cf316f37c155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edde4704253a4439bd7ca50b26e24e40, user_id=9abdce8004e24e81bd88bee757d8ec93, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.973018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:27.973547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.973888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.974136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:27.974861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:27.975172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:27.975778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4faedebd-8cdc-4a45-a338-cf316f37c155 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.976784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:27.983946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:27.992696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 789/1562] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:27.992696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:27.995321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 773/1562] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:27 2022] PUT /identity/v3/projects/edde4704253a4439bd7ca50b26e24e40/users/9abdce8004e24e81bd88bee757d8ec93/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:27.996698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:27.996991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:27.997306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.002865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54612782-91a8-45e5-98d0-1849025c950b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.015326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.016529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.057766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.058427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.058711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.058987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.059763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:28.060121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.061838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fa2d83-0320-4cc8-983e-1ffc67fec0b4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.073851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 790/1563] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:28.075350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.075497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.075667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 774/1564] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.076118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.080403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.090409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4420f7-6bc5-4ecd-b9e8-75a422b7a1c3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.094803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.094963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.095644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.095932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.096042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.096207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.096891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.099414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.104379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 791/1565] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.105215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.115992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cfca90-cb11-4907-9a1e-eb5ec9cc1965 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.120823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.121478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.121764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.122042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.122929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:28.123070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 775/1566] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 294 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.123305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.123787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.124894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78c776-f367-442b-8da3-d110548f9b1c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.133952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eb1de2-1105-4224-b01a-a540ebe205df admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.137881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 792/1567] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:28.138516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.139156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.139468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.139743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.140462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:28.140770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.141784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:41:28.141784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:41:28.141784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:41:28.142662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.145610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.145908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.146187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.155832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 776/1568] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:28.156646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.165912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:28.167108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:28.171175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02633041-9414-48be-ac4e-a7a2d763dda5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.178857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 793/1569] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:28.179379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.179482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.179581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.179581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.179581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.179581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.180443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.189505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a596e4b-e6f7-492c-a9a7-3678ae670b3a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.195342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.196019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.196357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.196634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.197415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:28.197727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.199394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.209274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 777/1570] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.210136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.220518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 794/1571] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:28.220518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.229900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae1eaf2-5698-443d-9056-2ba9b217c991 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.269585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0953c169-6969-4136-801b-12802cc6548a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.274143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.274792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.275074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.275382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.276070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.278129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.293088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.293756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.294052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.294315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.295200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:28.295560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.297315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.306063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 778/1572] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.306881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.318738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 795/1573] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:28.318738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.322475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e194db7e-0c50-4e21-afc9-bc6bc34bd1bd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.346691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.362146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.376837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f927e5a6-efae-4578-81e0-ed345239aec2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.381943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.382157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.382610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.382719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.382896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.383003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.383171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.383313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.383915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.384030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.385508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.385805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.395260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 779/1574] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:28.396814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.407126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0975d5-6663-4a20-9bbc-91485f0c5990 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.412428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.413102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.413378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.413657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.414336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.415184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 796/1575] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 389 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.416097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.416525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.445198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 780/1576] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.445914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.475440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0016581f-f6a3-4212-89c4-3a1728837c72 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.509601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00905e-a4dd-4d12-ac77-cdc1b6018a72 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.514219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.514886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.515170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.515507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.516200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.518142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.524124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.524875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.525162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.525462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.535511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.537538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.538479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 781/1577] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.539366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.547415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 797/1578] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:28.548203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.548915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31384b2-3713-48a3-a428-0ab8cfc85168 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.554249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.554986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.555370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.555714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.556475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.557850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365d489-bcb6-4140-b7a2-a97a818a7ab0 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.558520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.562924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.563344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.564075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.564075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.564576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.566470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.591179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 782/1579] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.591179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.591609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 798/1580] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.592203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.604338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3028d-0994-4f59-b834-c903a143911f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.669882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.670568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.670883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.671225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.671864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd076c6-a564-412e-b1e5-80336b4523d0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.671989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.673606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.676866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.677560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.677846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.678126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.678918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.680964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.685983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 799/1581] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:28.685983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.698384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.704810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c54e7-acd5-4860-8f5e-f051d906111e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.707971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:28.712955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.713746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.714318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.714654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.715400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.717782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.719056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 783/1582] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/projects => generated 403 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.719773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.748345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 800/1583] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 295 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.748345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.761892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2d3b0a-68d3-4639-80eb-6b64a1f8002b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.796451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a916a-30ba-4d06-a632-8a8e318b58cc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.807106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.808026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.808367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.808864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.809682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.812960 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.829860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.830578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.830858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.831137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.832155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.833827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.843251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 784/1584] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.844329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.848539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 801/1585] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:28.852384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.857848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1740a-22d6-4799-baa3-f5f12c55e3c9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.863433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed8f610-6c6b-40f0-b36a-5a03973a07e5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.863864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.865292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.865634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.866047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.866633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.868925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.871111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.871826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.872110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.872389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.873259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.875012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.889743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 802/1586] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:28.890852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.895974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 785/1587] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 292 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.897045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.905734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56de6562-7fce-47ee-8b47-e28f16c8c1ff admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.910107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60dabf1-ba8f-4a11-a410-4894fe430f1a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:28.912096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.913066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.913543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.913898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] PATH_INFO: `/v3/projects/150883a8e55c459a9f41fc48f8f81b0c/users/12f0ce0a293f4114a90098985e5f21ee/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.916907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:28.918721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:28.919324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:28.920012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:28.920913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.925071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.933242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=150883a8e55c459a9f41fc48f8f81b0c, user_id=12f0ce0a293f4114a90098985e5f21ee, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:28.935083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:41:28.935083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:41:28.935083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:41:28.937892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:28.957154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 803/1588] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:28 2022] PUT /identity/v3/projects/150883a8e55c459a9f41fc48f8f81b0c/users/12f0ce0a293f4114a90098985e5f21ee/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:28.957844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.960290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 786/1589] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:28 2022] POST /identity/v3/users => generated 302 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:28.961341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:28.975484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662c41bd-e7c5-4c00-a2a5-69e5144f3452 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.041510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee2bfc-1eb5-4659-a5c4-e99f17ccea2a admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.046298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.047661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.048169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.048550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.049376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.051084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.052767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.053629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.053764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.054389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.054795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.056518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.068690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 804/1590] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.068690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.069009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 787/1591] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.069009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.080224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595aa5d9-cebe-43c3-8c46-a11dfdd45f4d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.080610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b5ce0-236a-454d-8ae0-2c566e8c570f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.084870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.085486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292a420-618a-459b-bcb7-0f03b833cd72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.085627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.085800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.086075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.086515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a420-618a-459b-bcb7-0f03b833cd72 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.086797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.086977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a420-618a-459b-bcb7-0f03b833cd72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.087381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292a420-618a-459b-bcb7-0f03b833cd72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.088211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292a420-618a-459b-bcb7-0f03b833cd72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.088416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.090121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292a420-618a-459b-bcb7-0f03b833cd72 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.100141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 788/1592] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.101210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.105733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 805/1593] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.106628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.107001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.107384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.114335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d9f7ca-8d29-4d28-b01e-f40b419d7cb5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.119739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.120405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.120683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.121174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.122033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.123549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.126215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.128010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.137038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 789/1594] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.137038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.139185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 806/1595] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:29.140037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.148849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a24dd3-377f-4e88-8931-6146b53865c8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.151245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a169f1-3c2e-40d5-afff-bd3dd0adc3b6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.154246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.154915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.155361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.155491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.156218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.157159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.157844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.158014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.158390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.158761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.159788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.166821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.173933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 790/1596] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.173933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.183450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 807/1597] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.185160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.187148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce4d0d-e58c-4a5e-961f-2a78b221963f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.192454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.192923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.193208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.193489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.194221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.195860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.201230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce444616-c679-496f-8b30-5da7cd6c6c91 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.205968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.206249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 791/1598] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.206839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.206941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.207144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.207568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] PATH_INFO: `/v3/projects/6b1d15dc76094127a7f84c79dbb3053c/users/f83e72cd0d7e47b6bbfc4ffe64b71922/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.217680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.221226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.223589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a590dee5-d923-480f-8b0f-97645a84a879 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.227125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1d15dc76094127a7f84c79dbb3053c, user_id=f83e72cd0d7e47b6bbfc4ffe64b71922, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.230110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.230999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.231936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.232188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.232517 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] PATH_INFO: `/v3/projects/6bfcf706146c47f6910682b1db03b0d4/users/c53bebde020e4dd5a1b636c187008ae2/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.250595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 808/1599] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/6b1d15dc76094127a7f84c79dbb3053c/users/f83e72cd0d7e47b6bbfc4ffe64b71922/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.252030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.273926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bfcf706146c47f6910682b1db03b0d4, user_id=c53bebde020e4dd5a1b636c187008ae2, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.276652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.296917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 792/1600] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/6bfcf706146c47f6910682b1db03b0d4/users/c53bebde020e4dd5a1b636c187008ae2/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.298012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.326543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a5431-9ad3-49f2-a692-b795c95b8b84 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.388830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f5f0f-8700-4d9c-a5f2-42dbcfcb8235 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.392920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.393580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.393872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.394154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.397139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.398263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.398705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.399056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.400060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.402160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.408439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.410305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.417941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 809/1601] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.419100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.423110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 793/1602] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:29.424002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.429494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823fe55c-3f43-4133-ad8a-9ff2daecfedb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.434477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.435186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2511fa35-6737-47f1-99d5-73e30dd55c6e service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.435920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.436400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.437005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.438082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.440092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.441028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743c59a-1524-4153-8bb5-0f95d812bf0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.442671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743c59a-1524-4153-8bb5-0f95d812bf0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.443199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743c59a-1524-4153-8bb5-0f95d812bf0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.443514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743c59a-1524-4153-8bb5-0f95d812bf0d admin admin] PATH_INFO: `/v3/projects/c2805dfef22b4026a5b2a7b897618661/users/7f09808bb18247a09d1d4110f4835637/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.452341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 810/1603] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:29.453353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.462204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743c59a-1524-4153-8bb5-0f95d812bf0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2805dfef22b4026a5b2a7b897618661, user_id=7f09808bb18247a09d1d4110f4835637, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.465214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743c59a-1524-4153-8bb5-0f95d812bf0d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.465526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac6a5ee-f141-4dbf-b064-a7f37d931166 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.479560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad634387-c49b-4897-a539-6a5666e3cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.480611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad634387-c49b-4897-a539-6a5666e3cd64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.481047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad634387-c49b-4897-a539-6a5666e3cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.481432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad634387-c49b-4897-a539-6a5666e3cd64 admin admin] PATH_INFO: `/v3/projects/720fe6eaea524e23b60da4aa4a4973e3/users/18303519eb03426ba068c044a352775d/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.483176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 794/1604] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/c2805dfef22b4026a5b2a7b897618661/users/7f09808bb18247a09d1d4110f4835637/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.484595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.484891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.485172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.511467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.513075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.525190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad634387-c49b-4897-a539-6a5666e3cd64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=720fe6eaea524e23b60da4aa4a4973e3, user_id=18303519eb03426ba068c044a352775d, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.529864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad634387-c49b-4897-a539-6a5666e3cd64 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.567172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 811/1605] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/720fe6eaea524e23b60da4aa4a4973e3/users/18303519eb03426ba068c044a352775d/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.567172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.567172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.567172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.605625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] MFA Rules not processed for user `c53bebde020e4dd5a1b636c187008ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.607714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.625084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 795/1606] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:29.626224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.640884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9878fff-6ad1-469a-9dff-79ea9c37e2d3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.666971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.705578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 812/1607] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:29.706617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.748348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91f5af-fd8a-493f-8f75-6a8cf5017751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.749492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91f5af-fd8a-493f-8f75-6a8cf5017751 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.749923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91f5af-fd8a-493f-8f75-6a8cf5017751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.750459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91f5af-fd8a-493f-8f75-6a8cf5017751 admin admin] PATH_INFO: `/v3/projects/0586fe42a68745ec86eaca054e20ffea/users/826d10a724bc4c34bb8807220898b175/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.754063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:29.775428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91f5af-fd8a-493f-8f75-6a8cf5017751 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0586fe42a68745ec86eaca054e20ffea, user_id=826d10a724bc4c34bb8807220898b175, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.779423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91f5af-fd8a-493f-8f75-6a8cf5017751 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.796812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901c6dd-a284-4355-a5f4-823d5058ed12 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:29.804322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 796/1608] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/0586fe42a68745ec86eaca054e20ffea/users/826d10a724bc4c34bb8807220898b175/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.807585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ab11b-6568-4700-9efc-bd71c1dbf967 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.807874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ab11b-6568-4700-9efc-bd71c1dbf967 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.808154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ab11b-6568-4700-9efc-bd71c1dbf967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.853567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-eb1ab11b-6568-4700-9efc-bd71c1dbf967 None None] MFA Rules not processed for user `7f09808bb18247a09d1d4110f4835637`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.855881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-eb1ab11b-6568-4700-9efc-bd71c1dbf967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.898050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:29.899136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.899548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.900294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] PATH_INFO: `/v3/projects/bdb5ed31be104e3f85e523a70a01a12e/users/350ebea7facb418582ca7f862d4aa24e/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.925508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdb5ed31be104e3f85e523a70a01a12e, user_id=350ebea7facb418582ca7f862d4aa24e, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:29.930764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:29.943925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 797/1609] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:29.945189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:29.945518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:29.945820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:29.966985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 813/1610] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:29 2022] PUT /identity/v3/projects/bdb5ed31be104e3f85e523a70a01a12e/users/350ebea7facb418582ca7f862d4aa24e/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:29.968564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:29.971025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:29.978658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] MFA Rules not processed for user `18303519eb03426ba068c044a352775d`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:29.980333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.070769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 798/1611] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:30.072054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.084900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b24018-24b1-40bc-ba3d-fef325f407d0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.094149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38c151-b58d-45a2-9ea5-6e6a5fab9ff8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.095069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38c151-b58d-45a2-9ea5-6e6a5fab9ff8 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.095397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38c151-b58d-45a2-9ea5-6e6a5fab9ff8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.095696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38c151-b58d-45a2-9ea5-6e6a5fab9ff8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.171655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.189139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d4fbe1-80d4-4879-95d3-c695de5b0b2a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.196782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.199989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.200489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.201072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.202131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.206853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.222531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.245866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38c151-b58d-45a2-9ea5-6e6a5fab9ff8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.249899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38c151-b58d-45a2-9ea5-6e6a5fab9ff8 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.265801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 814/1612] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:30.267735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04ba59-d446-4a24-be65-5927eada29a1 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.268599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04ba59-d446-4a24-be65-5927eada29a1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.269063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04ba59-d446-4a24-be65-5927eada29a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.283006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 799/1613] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/projects => generated 385 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.293372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.324661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-bd04ba59-d446-4a24-be65-5927eada29a1 None None] MFA Rules not processed for user `826d10a724bc4c34bb8807220898b175`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:30.326517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bd04ba59-d446-4a24-be65-5927eada29a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.413156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 815/1614] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:30.414157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555206a3-18b8-4a13-a142-c4dbaa308bc0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.418618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555206a3-18b8-4a13-a142-c4dbaa308bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.419077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555206a3-18b8-4a13-a142-c4dbaa308bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.451202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3b1a8-6bc5-4231-83a3-340788302df8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.467456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed9de9-ee0d-4688-8954-cd5e2a915989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.467937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-555206a3-18b8-4a13-a142-c4dbaa308bc0 None None] MFA Rules not processed for user `350ebea7facb418582ca7f862d4aa24e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:30.468351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9de9-ee0d-4688-8954-cd5e2a915989 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.468652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9de9-ee0d-4688-8954-cd5e2a915989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.468939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed9de9-ee0d-4688-8954-cd5e2a915989 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.469689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed9de9-ee0d-4688-8954-cd5e2a915989 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.470264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-555206a3-18b8-4a13-a142-c4dbaa308bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.472062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed9de9-ee0d-4688-8954-cd5e2a915989 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.535305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 800/1615] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/projects => generated 389 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.535305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b1e4a-c99e-4fd9-84fa-504230b2891f None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.535305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b1e4a-c99e-4fd9-84fa-504230b2891f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.535305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b1e4a-c99e-4fd9-84fa-504230b2891f None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.540013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 801/1616] 10.222.0.43 () {54 vars in 961 bytes} [Fri Jun 24 08:41:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:30.540831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b1e4a-c99e-4fd9-84fa-504230b2891f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.656967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 816/1617] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:30.656967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1059f4-139b-4d52-b68a-587c63db534c None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.656967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1059f4-139b-4d52-b68a-587c63db534c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.656967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1059f4-139b-4d52-b68a-587c63db534c None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.656967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 817/1618] 10.222.0.43 () {54 vars in 961 bytes} [Fri Jun 24 08:41:30 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:30.656967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1059f4-139b-4d52-b68a-587c63db534c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:30.794247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b1e4a-c99e-4fd9-84fa-504230b2891f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.811155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92bfbfc-53dd-486c-b70a-63e4559d6058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.812172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bfbfc-53dd-486c-b70a-63e4559d6058 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.815335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bfbfc-53dd-486c-b70a-63e4559d6058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.815977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92bfbfc-53dd-486c-b70a-63e4559d6058 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.817062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bfbfc-53dd-486c-b70a-63e4559d6058 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.824579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92bfbfc-53dd-486c-b70a-63e4559d6058 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.860841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 802/1619] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/users => generated 299 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:30.863762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ec3f1-ec93-4e01-9a7b-23821c40931b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.864881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ec3f1-ec93-4e01-9a7b-23821c40931b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.865463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1ec3f1-ec93-4e01-9a7b-23821c40931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.880819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.893841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1f1ec3f1-ec93-4e01-9a7b-23821c40931b None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:30.895589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1f1ec3f1-ec93-4e01-9a7b-23821c40931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:30.924717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1059f4-139b-4d52-b68a-587c63db534c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:30.938388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:30.939146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:30.939459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:30.939737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:30.940445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:30.948251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:30.993348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:30.995717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 818/1620] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/users => generated 301 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:31.000420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.013418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 803/1621] 10.222.0.43 () {58 vars in 1058 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:31.014829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.015126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.015627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.032536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adc51e6-d963-4394-9e8d-01fd999c6b68 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.041109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:31.044988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:31.047900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.048662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.049023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.049377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.050155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.056591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.062325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 804/1622] 10.222.0.43 () {58 vars in 1058 bytes} [Fri Jun 24 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:31.063584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.076574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357a9fa-5362-4a75-b2b7-72ab69cf40f9 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.083839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.084547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.084946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.085295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.086053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.087835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.106413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 805/1623] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:31.107488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.119984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8601a7d2-80ea-4323-9efc-2ee35e11caf6 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.120603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 819/1624] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:30 2022] POST /identity/v3/projects => generated 401 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:31.122480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.243037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.245065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.245917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.246586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.248571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.250710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.261675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53717a-92ab-47c2-8124-5ab1911017c4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.268607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.269444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.269752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.270093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.271700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 806/1625] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:31.272851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.292629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ba78e8-c440-4b22-bc38-9566ebf34ae8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.302759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.303913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.305120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.305666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.372385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.383431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.387854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.390913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.399529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 820/1626] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:31.400681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.403656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 807/1627] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:31.404673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.417051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999d7871-3271-4596-9961-c1dad76a0aeb service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.421709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb7368-441c-4237-9df1-12242d457856 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.423492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.424615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.425056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.425664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] PATH_INFO: `/v3/projects/0863eba7b6434b73b6f6868a86d4c6f0/users/2c535b8170af4b17988b7bcbef1c96dc/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.427146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.428094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.429245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.430750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.431755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.433933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.446455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0863eba7b6434b73b6f6868a86d4c6f0, user_id=2c535b8170af4b17988b7bcbef1c96dc, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.450276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.472507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 808/1628] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:31 2022] POST /identity/v3/users => generated 307 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:31.474319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.487557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f6802c-6410-44f6-9995-ca62d9192e5c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.492400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 821/1629] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:31 2022] PUT /identity/v3/projects/0863eba7b6434b73b6f6868a86d4c6f0/users/2c535b8170af4b17988b7bcbef1c96dc/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:31.497541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.562446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:31.578025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:31.592563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.593548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.594364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.594484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] PATH_INFO: `/v3/projects/f1e8799d479c4cfcbe463be004486689/users/faaa13b2ae694ff0a63b16d1393fed58/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.614962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e8799d479c4cfcbe463be004486689, user_id=faaa13b2ae694ff0a63b16d1393fed58, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.617950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.636062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2154e6-084f-4c34-80ae-2e04dd35ff87 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.646955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 809/1630] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:31 2022] PUT /identity/v3/projects/f1e8799d479c4cfcbe463be004486689/users/faaa13b2ae694ff0a63b16d1393fed58/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:31.652310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.768051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b4ae16-47b7-45b4-af0f-35d554fb628d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.780792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.781537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.781827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.782108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.782963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.787546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.794751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.795711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.796193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.796743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.797642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.799443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.812787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 810/1631] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:31.813779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.827458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b791db0c-6c79-4308-8142-74f40a93ae30 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.828209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 822/1632] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:31.829194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.837389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.838251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.838544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.838879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.843423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.845127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.847172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3ba5b-ac8c-482e-a49b-38f871605318 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.860713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 811/1633] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:31.862076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.873524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f432ce-8c8c-4a20-b853-0a1f57fb2544 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:31.878385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.878385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.878385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.878385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] PATH_INFO: `/v3/projects/0863eba7b6434b73b6f6868a86d4c6f0/users/2c535b8170af4b17988b7bcbef1c96dc/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.886157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8997fe03-addd-44dc-a709-a1dbd773df63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:31.889964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997fe03-addd-44dc-a709-a1dbd773df63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.890301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997fe03-addd-44dc-a709-a1dbd773df63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.890571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8997fe03-addd-44dc-a709-a1dbd773df63 admin admin] PATH_INFO: `/v3/projects/9d415ae4f57a459d97507abd7584d88a/users/324ce6b583264a4ba67c35f229b89586/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:31.891409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0863eba7b6434b73b6f6868a86d4c6f0, user_id=2c535b8170af4b17988b7bcbef1c96dc, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.901905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.930529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997fe03-addd-44dc-a709-a1dbd773df63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d415ae4f57a459d97507abd7584d88a, user_id=324ce6b583264a4ba67c35f229b89586, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:31.931088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 823/1634] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:31 2022] PUT /identity/v3/projects/0863eba7b6434b73b6f6868a86d4c6f0/users/2c535b8170af4b17988b7bcbef1c96dc/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:31.932824 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:31.949893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8997fe03-addd-44dc-a709-a1dbd773df63 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:31.978202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 812/1635] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:31 2022] PUT /identity/v3/projects/9d415ae4f57a459d97507abd7584d88a/users/324ce6b583264a4ba67c35f229b89586/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:31.986227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6b19b-2382-4e14-947c-04313d09097f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:31.986227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6b19b-2382-4e14-947c-04313d09097f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:31.986547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6b19b-2382-4e14-947c-04313d09097f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.044429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1037f9b-a4a8-4ed7-bf48-6f6975094992 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.052572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-26e6b19b-2382-4e14-947c-04313d09097f None None] MFA Rules not processed for user `2c535b8170af4b17988b7bcbef1c96dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.054425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-26e6b19b-2382-4e14-947c-04313d09097f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.118462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.119724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.120151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.120524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] PATH_INFO: `/v3/projects/f1e8799d479c4cfcbe463be004486689/users/faaa13b2ae694ff0a63b16d1393fed58/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.123075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e8799d479c4cfcbe463be004486689, user_id=faaa13b2ae694ff0a63b16d1393fed58, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.125994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.134232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:32.146867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 824/1636] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:31 2022] PUT /identity/v3/projects/f1e8799d479c4cfcbe463be004486689/users/faaa13b2ae694ff0a63b16d1393fed58/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:32.156937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.188468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 813/1637] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:32.190438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab97cd-89b6-4135-a3f8-43459a1c1240 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.191167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab97cd-89b6-4135-a3f8-43459a1c1240 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.191563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab97cd-89b6-4135-a3f8-43459a1c1240 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.194350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 814/1638] 10.222.0.43 () {54 vars in 912 bytes} [Fri Jun 24 08:41:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:32.203097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5e561-9b96-4e83-8f8f-38f136bf3ee3 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.203817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5e561-9b96-4e83-8f8f-38f136bf3ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.204239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5e561-9b96-4e83-8f8f-38f136bf3ee3 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.206113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 815/1639] 10.222.0.43 () {54 vars in 912 bytes} [Fri Jun 24 08:41:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:32.207681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.208059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.209099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.259984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] MFA Rules not processed for user `faaa13b2ae694ff0a63b16d1393fed58`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.269265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.274513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5325fdc-ef4c-4cf0-a0a9-4aa90e7a1c2e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.280420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa73a228-d33d-42bf-abcd-0974db9056e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.281170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a228-d33d-42bf-abcd-0974db9056e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.281456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a228-d33d-42bf-abcd-0974db9056e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.281891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa73a228-d33d-42bf-abcd-0974db9056e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.282650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a228-d33d-42bf-abcd-0974db9056e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.284755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa73a228-d33d-42bf-abcd-0974db9056e2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.288521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:32.303835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 825/1640] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:32.303835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19356219-4070-4d46-b244-73b622abbdc3 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.304762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19356219-4070-4d46-b244-73b622abbdc3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.305121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19356219-4070-4d46-b244-73b622abbdc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.339243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-19356219-4070-4d46-b244-73b622abbdc3 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.340861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-19356219-4070-4d46-b244-73b622abbdc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.414956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 816/1641] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:32.415876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.441968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 826/1642] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:32.451093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.451093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.452061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.495344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.499353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.514022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 827/1643] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:32.515469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.531561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a3c1f-ae3f-4199-9c3b-322c243e2380 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.540729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.543905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.544213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.544501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] PATH_INFO: `/v3/projects/9d415ae4f57a459d97507abd7584d88a/users/324ce6b583264a4ba67c35f229b89586/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.547455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d415ae4f57a459d97507abd7584d88a, user_id=324ce6b583264a4ba67c35f229b89586, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.548728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6dc592-c54a-4e90-a40d-42a6fdaad796 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.552760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.554315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5c654-9e81-44af-bac1-db460a4c881c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.555007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5c654-9e81-44af-bac1-db460a4c881c service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.555383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5c654-9e81-44af-bac1-db460a4c881c service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.555725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5c654-9e81-44af-bac1-db460a4c881c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.579602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 828/1644] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:32 2022] PUT /identity/v3/projects/9d415ae4f57a459d97507abd7584d88a/users/324ce6b583264a4ba67c35f229b89586/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:32.581408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.634318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5c654-9e81-44af-bac1-db460a4c881c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.636595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5c654-9e81-44af-bac1-db460a4c881c service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.653830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 817/1645] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:32.655156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.655477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.655792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.680474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ff70d-5f87-4d68-80f4-465031d16aeb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.685608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.686589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.687196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.687954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.716392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:32.741304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] MFA Rules not processed for user `324ce6b583264a4ba67c35f229b89586`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:32.744640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:32.809513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.810988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:32.824999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 829/1646] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:32.826062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.835659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be1ae2c-16b2-4946-9a12-8344c5ac3c4a service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.844496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.844985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:32.845323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.848586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.848943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.881543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 818/1647] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:32.882744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:32.896402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510aa847-092e-4214-bdbc-0d27129918ed None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:32.903278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:32.905065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:32.905547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:32.906003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:32.979700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:32.981943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.010424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:33.012460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.013172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 830/1648] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:33.027762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 819/1649] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:33.256406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:33.308484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:33.319929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35de37a6-c35d-4713-bda8-b380d88e3f84 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:33.324157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:33.325020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:33.325481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:33.325984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:33.352213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:33.361371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.386812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 820/1650] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:33 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:33.388294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf160376-c76e-42a2-a7fb-3dc0e511a226 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:33.394140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:33.395025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:33.395338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:33.396169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] PATH_INFO: `/v3/users/c53bebde020e4dd5a1b636c187008ae2` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:33.398158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c53bebde020e4dd5a1b636c187008ae2)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:33.400994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: WARNING py.warnings [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:41:33.400994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: warnings.warn(msg) Jun 24 08:41:33.400994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: Jun 24 08:41:33.402740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.527611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:33.531501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c53bebde020e4dd5a1b636c187008ae2'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.557654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c53bebde020e4dd5a1b636c187008ae2'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.569029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c53bebde020e4dd5a1b636c187008ae2'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.570931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c53bebde020e4dd5a1b636c187008ae2'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.571674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d519f0d-0847-4d76-a234-ce55348614cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c53bebde020e4dd5a1b636c187008ae2'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.654210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:33.655551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 831/1651] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:33 2022] DELETE /identity/v3/users/c53bebde020e4dd5a1b636c187008ae2 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:33.659614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:33.659907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:33.660796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:33.673238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9d0e1b-a65c-49ad-b37e-39c51816ee2e service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:33.698612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:33.700463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:33.837209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:33.838457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:33.838760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:33.839101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] PATH_INFO: `/v3/users/18303519eb03426ba068c044a352775d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:33.840330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=18303519eb03426ba068c044a352775d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:33.842578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:33.847346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 832/1652] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:33.861302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:33.872511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:33.958981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18303519eb03426ba068c044a352775d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.976286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18303519eb03426ba068c044a352775d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:33.982042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:34.010323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18303519eb03426ba068c044a352775d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.025746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18303519eb03426ba068c044a352775d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.027192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18303519eb03426ba068c044a352775d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.030698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95c934c-a4a4-4e64-af67-6e354b6f31c6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.041382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 821/1653] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:33 2022] DELETE /identity/v3/users/18303519eb03426ba068c044a352775d => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:34.042413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.043818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.048039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.061138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.061526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.061702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.079164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.116421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.118100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.150699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7955f4d-9b13-4b00-8eef-e48d60ba6b5f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.153215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 833/1654] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:34.154446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.156457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.167097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.168053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.168341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.168621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] PATH_INFO: `/v3/users/350ebea7facb418582ca7f862d4aa24e` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.170571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] RBAC: Authorizing `identity:delete_user(user_id=350ebea7facb418582ca7f862d4aa24e)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.172582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.184438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152aef4f-1108-4685-a62e-e9fbb88952c0 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.201457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.202364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.202649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.202924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] PATH_INFO: `/v3/users/7f09808bb18247a09d1d4110f4835637` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.208225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f09808bb18247a09d1d4110f4835637)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.214551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.277839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '350ebea7facb418582ca7f862d4aa24e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.304360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '350ebea7facb418582ca7f862d4aa24e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.337407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '350ebea7facb418582ca7f862d4aa24e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.337581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f09808bb18247a09d1d4110f4835637'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.359137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f09808bb18247a09d1d4110f4835637'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.359932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '350ebea7facb418582ca7f862d4aa24e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.362355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ce454f69-a0c9-4c60-9f3e-e460d8e1879a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '350ebea7facb418582ca7f862d4aa24e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.365388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:34.374360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f09808bb18247a09d1d4110f4835637'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.375629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f09808bb18247a09d1d4110f4835637'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.376525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1bec91c7-deae-4eac-8a2d-0b5dfa3aa362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f09808bb18247a09d1d4110f4835637'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:34.378407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 822/1655] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:34 2022] DELETE /identity/v3/users/350ebea7facb418582ca7f862d4aa24e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:34.381623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e7af-671f-4c19-83f0-347fef97eb18 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.382236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e7af-671f-4c19-83f0-347fef97eb18 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.383767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d565e7af-671f-4c19-83f0-347fef97eb18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.427737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 834/1656] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:34 2022] DELETE /identity/v3/users/7f09808bb18247a09d1d4110f4835637 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:34.429286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.429582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.431094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.433788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-d565e7af-671f-4c19-83f0-347fef97eb18 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:34.435029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d565e7af-671f-4c19-83f0-347fef97eb18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.485615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:34.491150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.576056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:34.611542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 823/1657] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:34.613820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.615113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.615752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.653921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 835/1658] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:34.658839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:34.663520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.692186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.694170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.714143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 824/1659] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:34.723476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.745808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:34.878172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c87bc1a-6a3c-4ce7-ab7c-3c0211040320 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.890612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.890612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.890612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.890612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.898113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728628c0-c232-4c26-a2c2-aaaf3d65c026 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.907204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.907766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.908049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.908326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:34.912251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.948733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.955061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:34.956870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:34.966641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 836/1660] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:34.967607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:34.984923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b62bca-634e-4aad-80a1-711636b9a3f8 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:34.985640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 825/1661] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:34.991802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:34.993321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:34.993616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:34.993894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:35.010698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:35.012249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:35.024198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 837/1662] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:36.061340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.063851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.090370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840c88e-4898-4a2d-aa5a-d69fffdbde66 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.103359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.105188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.105481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.105765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] PATH_INFO: `/v3/projects/bdb5ed31be104e3f85e523a70a01a12e` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.109703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdb5ed31be104e3f85e523a70a01a12e)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.110905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: WARNING py.warnings [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 24 08:41:36.110905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: warnings.warn(msg) Jun 24 08:41:36.110905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: Jun 24 08:41:36.112482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.208045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdb5ed31be104e3f85e523a70a01a12e', 'request_id': 'req-2e1531ea-202b-42cb-b93d-f338aadc6d50'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.209649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.211150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.230973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b266e0b8-8867-4ac8-a7bf-426cecde8cac service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.232190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdb5ed31be104e3f85e523a70a01a12e', 'request_id': 'req-2e1531ea-202b-42cb-b93d-f338aadc6d50'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.276067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdb5ed31be104e3f85e523a70a01a12e', 'request_id': 'req-2e1531ea-202b-42cb-b93d-f338aadc6d50'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.311442 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] The token cache is being invalidate because project bdb5ed31be104e3f85e523a70a01a12e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:36.312494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.313693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.316375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 826/1663] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:36 2022] DELETE /identity/v3/projects/bdb5ed31be104e3f85e523a70a01a12e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:36.322088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.325581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.384054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.384959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.385244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.385529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] PATH_INFO: `/v3/projects/6bfcf706146c47f6910682b1db03b0d4` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.390512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bfcf706146c47f6910682b1db03b0d4)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.393106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.408748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:36.465355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bfcf706146c47f6910682b1db03b0d4', 'request_id': 'req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.471539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bfcf706146c47f6910682b1db03b0d4', 'request_id': 'req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.479774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bfcf706146c47f6910682b1db03b0d4', 'request_id': 'req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.501696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] The token cache is being invalidate because project 6bfcf706146c47f6910682b1db03b0d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:36.502306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.503278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.507623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 838/1664] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:36 2022] DELETE /identity/v3/projects/6bfcf706146c47f6910682b1db03b0d4 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:36.511387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:36.513065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:36.517035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1531ea-202b-42cb-b93d-f338aadc6d50 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.555493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:36.635966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf287b8e-fbc4-4ea3-8143-bf75126b00c4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:36.643112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.645638 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.647180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.647752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] PATH_INFO: `/v3/users/2c535b8170af4b17988b7bcbef1c96dc` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.651690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c535b8170af4b17988b7bcbef1c96dc)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.654811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.665522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:36.666499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:36.669596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:36.669908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] PATH_INFO: `/v3/users/324ce6b583264a4ba67c35f229b89586` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:36.672134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] RBAC: Authorizing `identity:delete_user(user_id=324ce6b583264a4ba67c35f229b89586)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:36.674501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:36.759818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c535b8170af4b17988b7bcbef1c96dc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.775869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c535b8170af4b17988b7bcbef1c96dc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.792450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c535b8170af4b17988b7bcbef1c96dc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.795693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c535b8170af4b17988b7bcbef1c96dc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.796845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c535b8170af4b17988b7bcbef1c96dc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.819682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '324ce6b583264a4ba67c35f229b89586'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.845037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '324ce6b583264a4ba67c35f229b89586'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.868870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 839/1665] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:36 2022] DELETE /identity/v3/users/2c535b8170af4b17988b7bcbef1c96dc => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:36.902998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '324ce6b583264a4ba67c35f229b89586'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.927952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '324ce6b583264a4ba67c35f229b89586'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.929678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '324ce6b583264a4ba67c35f229b89586'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:36.948811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 827/1666] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:36 2022] DELETE /identity/v3/users/324ce6b583264a4ba67c35f229b89586 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:37.119577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:37.119995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.160920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:37.341503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c225e2-bdfe-47c1-a80e-94f598bdc029 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.474376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.475606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.476015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.476400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] PATH_INFO: `/v3/projects/c2805dfef22b4026a5b2a7b897618661` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.481170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2805dfef22b4026a5b2a7b897618661)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.483330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.604081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2805dfef22b4026a5b2a7b897618661', 'request_id': 'req-e609bc18-dabc-45e1-afb8-71eadb5c4fae'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.623375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2805dfef22b4026a5b2a7b897618661', 'request_id': 'req-e609bc18-dabc-45e1-afb8-71eadb5c4fae'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.638782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2805dfef22b4026a5b2a7b897618661', 'request_id': 'req-e609bc18-dabc-45e1-afb8-71eadb5c4fae'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.667274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] The token cache is being invalidate because project c2805dfef22b4026a5b2a7b897618661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:37.667274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.667793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.671597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 840/1667] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:37 2022] DELETE /identity/v3/projects/c2805dfef22b4026a5b2a7b897618661 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:37.675724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:37.677832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.703641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2c62a-5904-44bb-8488-a6d2fb8cb76c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.757951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:37.759538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:37.848713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.849867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.850227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.850579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] PATH_INFO: `/v3/users/faaa13b2ae694ff0a63b16d1393fed58` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.851996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=faaa13b2ae694ff0a63b16d1393fed58)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.854961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.889716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:37.915049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609bc18-dabc-45e1-afb8-71eadb5c4fae admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:37.926398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:37.932912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:37.933251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:37.936289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] PATH_INFO: `/v3/projects/720fe6eaea524e23b60da4aa4a4973e3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:37.939273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] RBAC: Authorizing `identity:delete_project(project_id=720fe6eaea524e23b60da4aa4a4973e3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:37.944047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:37.951024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:37.977406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faaa13b2ae694ff0a63b16d1393fed58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:37.990014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faaa13b2ae694ff0a63b16d1393fed58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.013034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faaa13b2ae694ff0a63b16d1393fed58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.029274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faaa13b2ae694ff0a63b16d1393fed58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.030894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0dc4f144-37cc-4d2c-b656-8717f930aca3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faaa13b2ae694ff0a63b16d1393fed58'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.048864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 828/1668] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:37 2022] DELETE /identity/v3/users/faaa13b2ae694ff0a63b16d1393fed58 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:38.100661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '720fe6eaea524e23b60da4aa4a4973e3', 'request_id': 'req-98f3d48c-f083-4d2a-906e-a765d7808085'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.115153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '720fe6eaea524e23b60da4aa4a4973e3', 'request_id': 'req-98f3d48c-f083-4d2a-906e-a765d7808085'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.127769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '720fe6eaea524e23b60da4aa4a4973e3', 'request_id': 'req-98f3d48c-f083-4d2a-906e-a765d7808085'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.178069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] The token cache is being invalidate because project 720fe6eaea524e23b60da4aa4a4973e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:38.185677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.187503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-98f3d48c-f083-4d2a-906e-a765d7808085 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:38.195303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 841/1669] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:37 2022] DELETE /identity/v3/projects/720fe6eaea524e23b60da4aa4a4973e3 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:38.200558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.200859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.201174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.236475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:38.237716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.395258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 829/1670] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:38.397962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.398427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.399138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.438038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:38.439587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.462964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 842/1671] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:38.464273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.479901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f577d-f319-4616-8749-cbcb16ed8066 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.493678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.495004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.495398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.495695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.496440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:38.496750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.498931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34f8131-e0f7-4d9a-af8a-59d6fc801a53 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.515249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 830/1672] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:38.519304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.540026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941f5762-57f5-46c2-b286-8e44097569e7 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.546479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.547356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.547712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.548059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.548839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.550843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.597741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 843/1673] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/projects => generated 405 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:38.602417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8f958-4150-40eb-b0d3-0798974402bb None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.603516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8f958-4150-40eb-b0d3-0798974402bb None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.605123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c8f958-4150-40eb-b0d3-0798974402bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.608285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.653488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-90c8f958-4150-40eb-b0d3-0798974402bb None None] MFA Rules not processed for user `9abdce8004e24e81bd88bee757d8ec93`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:38.655024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-90c8f958-4150-40eb-b0d3-0798974402bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.757121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3fc48-001e-45b4-85a2-24f4eaf47520 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.763012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.764128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.764639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.765103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.765998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.768164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.799994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 844/1674] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/users => generated 303 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:38.804554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.823985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cd5f65-df08-4304-b47b-a798770fbfc4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.828518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 831/1675] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:38.840858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.840858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.842180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.842464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.842820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.843559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.843841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.844649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.846897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.873387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 845/1676] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:38.874365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.879012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] MFA Rules not processed for user `9abdce8004e24e81bd88bee757d8ec93`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:38.885115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.892984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7986ef-15ae-4b89-ab2a-09a55ef5db4b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.898816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 832/1677] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:38.907407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.907407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.907764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.907764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.909160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:38.909716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.911771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:38.914913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:38.942801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 846/1678] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:38.943779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:38.966225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf20af-7a1f-429f-9593-2858fb64ced9 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:38.976807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6640c858-9527-4cc4-ab2b-1726ec993937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:38.977739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c858-9527-4cc4-ab2b-1726ec993937 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:38.978024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c858-9527-4cc4-ab2b-1726ec993937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:38.978303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6640c858-9527-4cc4-ab2b-1726ec993937 admin admin] PATH_INFO: `/v3/projects/78a138fb930741858ff86e0322818521/users/d513f970536a4876b16d3e122967dbd4/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:38.987925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:38.998772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640c858-9527-4cc4-ab2b-1726ec993937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a138fb930741858ff86e0322818521, user_id=d513f970536a4876b16d3e122967dbd4, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.001687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6640c858-9527-4cc4-ab2b-1726ec993937 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.013798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=87,keystone:DELETE=7,keystone:INSERT=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:39.019928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 847/1679] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:38 2022] PUT /identity/v3/projects/78a138fb930741858ff86e0322818521/users/d513f970536a4876b16d3e122967dbd4/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:39.030757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.031329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.031772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.054423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.056060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.111063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985ab4c5-5608-4992-92f4-e96f6e3684fc None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.148174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 848/1680] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:39.157060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.159305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.246842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.247545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.247832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.248107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.259323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de2ed0-41ac-497b-b451-f41fb329abda None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.264291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.265219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.265733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.266226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.299026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.314924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.329154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 849/1681] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:39.330857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.333153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.370248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f01fdb-d59b-4445-81f8-81e40b77df99 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.388863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.389775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.390068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.390345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] PATH_INFO: `/v3/projects/f1e8799d479c4cfcbe463be004486689` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.393892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e8799d479c4cfcbe463be004486689)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.396083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.463107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.474317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.519290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 833/1682] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:39.520528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.527155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.530240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e8799d479c4cfcbe463be004486689', 'request_id': 'req-41e988da-6ec5-41c1-b56a-4bd8947359c4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.537500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e8799d479c4cfcbe463be004486689', 'request_id': 'req-41e988da-6ec5-41c1-b56a-4bd8947359c4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.544984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e8799d479c4cfcbe463be004486689', 'request_id': 'req-41e988da-6ec5-41c1-b56a-4bd8947359c4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.561075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce25ec0-6cd2-4e0d-b8b7-d38dedfd2537 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.591840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] The token cache is being invalidate because project f1e8799d479c4cfcbe463be004486689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:39.592459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.594664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.603136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 850/1683] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:39 2022] DELETE /identity/v3/projects/f1e8799d479c4cfcbe463be004486689 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:39.603136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:39.608913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.688851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:39.742859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.743681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.743968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.744252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] PATH_INFO: `/v3/projects/0863eba7b6434b73b6f6868a86d4c6f0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.745537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=0863eba7b6434b73b6f6868a86d4c6f0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.751736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.766270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:39.787353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e988da-6ec5-41c1-b56a-4bd8947359c4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:39.794719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:39.796060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.796541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.796998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] PATH_INFO: `/v3/projects/9d415ae4f57a459d97507abd7584d88a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.821023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d415ae4f57a459d97507abd7584d88a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:39.827948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:39.857522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0863eba7b6434b73b6f6868a86d4c6f0', 'request_id': 'req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.866497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0863eba7b6434b73b6f6868a86d4c6f0', 'request_id': 'req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.886859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0863eba7b6434b73b6f6868a86d4c6f0', 'request_id': 'req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.914921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d415ae4f57a459d97507abd7584d88a', 'request_id': 'req-87df92d4-5433-4dac-875b-c95813e96b14'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.917677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] The token cache is being invalidate because project 0863eba7b6434b73b6f6868a86d4c6f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:39.917983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.920073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d9a89aef-e9ba-4d66-94cb-44ce48137ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.925095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d415ae4f57a459d97507abd7584d88a', 'request_id': 'req-87df92d4-5433-4dac-875b-c95813e96b14'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.929968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 834/1684] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:39 2022] DELETE /identity/v3/projects/0863eba7b6434b73b6f6868a86d4c6f0 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:39.931262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f61f4-583a-4094-85a0-bd1d5ba54639 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.931562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f61f4-583a-4094-85a0-bd1d5ba54639 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.933017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f61f4-583a-4094-85a0-bd1d5ba54639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.933983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d415ae4f57a459d97507abd7584d88a', 'request_id': 'req-87df92d4-5433-4dac-875b-c95813e96b14'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.952529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] The token cache is being invalidate because project 9d415ae4f57a459d97507abd7584d88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:39.953309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.953839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87df92d4-5433-4dac-875b-c95813e96b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:39.959542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 851/1685] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:39 2022] DELETE /identity/v3/projects/9d415ae4f57a459d97507abd7584d88a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:39.959542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e5b89-ed5a-4aa5-92b0-ca70fe49905f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:39.959542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e5b89-ed5a-4aa5-92b0-ca70fe49905f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:39.959981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e5b89-ed5a-4aa5-92b0-ca70fe49905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:39.977252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-8d0f61f4-583a-4094-85a0-bd1d5ba54639 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.978515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8d0f61f4-583a-4094-85a0-bd1d5ba54639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:39.988226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-806e5b89-ed5a-4aa5-92b0-ca70fe49905f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:39.990085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-806e5b89-ed5a-4aa5-92b0-ca70fe49905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.080224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 852/1686] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:40.081641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407475cb-006e-43e4-844a-b547697882fb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.081938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407475cb-006e-43e4-844a-b547697882fb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.082215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407475cb-006e-43e4-844a-b547697882fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.119738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-407475cb-006e-43e4-844a-b547697882fb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.120977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-407475cb-006e-43e4-844a-b547697882fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.139956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 853/1687] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:40.147144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a07a8-d55b-439c-ab3f-db64e4ddb7c6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.147463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a07a8-d55b-439c-ab3f-db64e4ddb7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.147750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9a07a8-d55b-439c-ab3f-db64e4ddb7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.174124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6c9a07a8-d55b-439c-ab3f-db64e4ddb7c6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.179388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6c9a07a8-d55b-439c-ab3f-db64e4ddb7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.179550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 835/1688] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:40.179550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.179699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.180102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.191309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 854/1689] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:40.193459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.193758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.194038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.219799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.220655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.230388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:40.231167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:40.249482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 855/1690] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:40.250526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.271709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63109fe4-9bae-40fe-80f1-ba6726b2ea84 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.272007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 836/1691] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:40.285692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.296901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.297747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.298037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.298462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.301446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:40.301873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.305062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.320794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4dcec-66b0-48a8-b026-adbb6c8b81a7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.337151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.337593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 856/1692] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:40.337862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.338174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.338423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.338608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.343375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:40.343764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.345426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.381315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 837/1693] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:40.382251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.393599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e77616-cc46-4dfc-873e-33a5f5ec079e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.407867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.409507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.409987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.410448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.411952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:40.415858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.418076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.433746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11d3c6-51e0-443b-bbc5-147b247fa190 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.439783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 857/1694] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:40.439783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.460056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.462611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.462779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fcbd5-446a-438c-b750-26bba4669b18 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.462899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.463168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.469916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.470751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.471158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.471560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.471679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.472444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.474115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.476590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.491430 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:40.544207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 858/1695] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:40.546197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.571597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 838/1696] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/projects => generated 409 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:40.572455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.734684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4689352c-0647-403c-8e1d-0c9c9eb5176a admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.749511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:40.806190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3b8c4-3658-49fb-9a93-68f721e08402 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.816507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.817208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.817563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.817845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.818557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.821186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.874019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 839/1697] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/users => generated 294 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:40.874596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:40.923732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d9a41-d123-49d1-b316-620d26cff5d7 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:40.932700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.935247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.936349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.936634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.939694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.942093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6bab59-3238-4e7b-b179-2b435b1b77d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:40.943513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bab59-3238-4e7b-b179-2b435b1b77d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:40.943737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bab59-3238-4e7b-b179-2b435b1b77d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:40.944018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6bab59-3238-4e7b-b179-2b435b1b77d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:40.944725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bab59-3238-4e7b-b179-2b435b1b77d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:40.951876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.953361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6bab59-3238-4e7b-b179-2b435b1b77d5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:40.993512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 840/1698] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/users => generated 305 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:41.001405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee275-1007-4898-8c92-170b0008c200 None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.001786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee275-1007-4898-8c92-170b0008c200 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.003391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ee275-1007-4898-8c92-170b0008c200 None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.010635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 841/1699] 10.222.0.43 () {54 vars in 912 bytes} [Fri Jun 24 08:41:40 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:41.011538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ee275-1007-4898-8c92-170b0008c200 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.033520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 859/1700] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:40 2022] POST /identity/v3/projects => generated 371 bytes in 488 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:41.037561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.065435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ee275-1007-4898-8c92-170b0008c200 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.219083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e3ec17-47ac-4ca7-8d84-4b89338ff50b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.223013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042ece-d6c2-4858-931a-d0b101f44368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.224729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042ece-d6c2-4858-931a-d0b101f44368 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.227401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042ece-d6c2-4858-931a-d0b101f44368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.227859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042ece-d6c2-4858-931a-d0b101f44368 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.231523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042ece-d6c2-4858-931a-d0b101f44368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.236256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042ece-d6c2-4858-931a-d0b101f44368 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.236611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.237305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.237587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.237865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.238642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.243541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.259468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 842/1701] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:41.265785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 860/1702] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:41.265785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.269439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.270131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.272074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.295795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17688511-2084-4c33-bdf7-d915e4f45ec1 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.306469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.318079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.319336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.321414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.322000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.325603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.358632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:41.371781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:41.375423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 861/1703] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:41 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:41.375564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:41.375654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.396809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d3525-5444-4f8b-9e01-28592e69a3bf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.409687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.412196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.412564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.412912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.413736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.416366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.448195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 862/1704] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:41.450427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.474417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbce13-999f-448b-a7bb-3d633e950a26 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.484423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 843/1705] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:41.485909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.494385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac82285-b152-49e6-9333-7197238ec640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.499124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e8c3b5-e3e0-4d9f-9c0a-b7e202d96a52 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.500002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac82285-b152-49e6-9333-7197238ec640 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.500489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac82285-b152-49e6-9333-7197238ec640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.500968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac82285-b152-49e6-9333-7197238ec640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.504512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac82285-b152-49e6-9333-7197238ec640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.506962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac82285-b152-49e6-9333-7197238ec640 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.513376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.514066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.514348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.515196 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.515976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.517679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.532190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 863/1706] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:41.533942 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.534444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.534978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.540694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 844/1707] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:41.541634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.553390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7e09a-041b-41ec-8e82-01be9c9ded7b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.558288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.559190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.559501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.559816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] PATH_INFO: `/v3/projects/465ad3d0ecd242d58f347fff5771468f/users/89371fb7d9c34dca8a81c5dfe6d2acd6/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.593583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=465ad3d0ecd242d58f347fff5771468f, user_id=89371fb7d9c34dca8a81c5dfe6d2acd6, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.597758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.620190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] MFA Rules not processed for user `d513f970536a4876b16d3e122967dbd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:41.622325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:41.626583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 845/1708] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:41 2022] PUT /identity/v3/projects/465ad3d0ecd242d58f347fff5771468f/users/89371fb7d9c34dca8a81c5dfe6d2acd6/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:41.628544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.641893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:41.681470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:41.741527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81969712-1352-4bce-bde6-0c488d4cd907 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.746987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.747796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.748200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.748501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.761549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.771997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.776438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 864/1709] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:41.777572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.784125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 846/1710] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:41.785171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:41.906228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecb81c1-b797-4345-832b-bd8d64078ffd service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.919899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6f0c30-d12b-43d3-833c-02d944438b5f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:41.920730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1db6a7-50da-4767-b1df-bf3f44aab7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.921442 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db6a7-50da-4767-b1df-bf3f44aab7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.921715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db6a7-50da-4767-b1df-bf3f44aab7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.921992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1db6a7-50da-4767-b1df-bf3f44aab7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.922727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1db6a7-50da-4767-b1df-bf3f44aab7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:41.924371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1db6a7-50da-4767-b1df-bf3f44aab7c3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:41.933490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:41.936078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.937280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.937838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] PATH_INFO: `/v3/projects/b1d8a1067b954225b585b6fc5b9b5843/users/5ebda67a890d4a64a0eb12ec67c7d3df/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:41.963873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 847/1711] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:41 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:41.965159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:41.967772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:41.968715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.050880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] MFA Rules not processed for user `89371fb7d9c34dca8a81c5dfe6d2acd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:42.053487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:42.149113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d8a1067b954225b585b6fc5b9b5843, user_id=5ebda67a890d4a64a0eb12ec67c7d3df, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.153076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:42.177219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 848/1712] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:42.178352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:42.184395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 865/1713] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:41 2022] PUT /identity/v3/projects/b1d8a1067b954225b585b6fc5b9b5843/users/5ebda67a890d4a64a0eb12ec67c7d3df/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:42.185787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:42.186940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:42.213545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1544b05c-51f6-44ab-a3c8-a0d8717b24a9 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:42.285818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:42.304750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:42.311730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee49b05-189b-4389-88fe-c96ccf8e828a admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:42.318790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:42.321109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.321626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.322116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] PATH_INFO: `/v3/projects/095b1d6ba4e448a7ac328185abbb784f/users/af57181073784691ba5562c83b65beea/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.332207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226f4d6-8730-491e-abd9-8de369e6133e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:42.333679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226f4d6-8730-491e-abd9-8de369e6133e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.333966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226f4d6-8730-491e-abd9-8de369e6133e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.334241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226f4d6-8730-491e-abd9-8de369e6133e admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.335311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226f4d6-8730-491e-abd9-8de369e6133e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.338004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226f4d6-8730-491e-abd9-8de369e6133e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:42.352709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=095b1d6ba4e448a7ac328185abbb784f, user_id=af57181073784691ba5562c83b65beea, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.358628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:42.395333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 866/1714] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:42 2022] PUT /identity/v3/projects/095b1d6ba4e448a7ac328185abbb784f/users/af57181073784691ba5562c83b65beea/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:42.397482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:42.406077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 849/1715] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/projects => generated 405 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:42.406077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecf4e0-5722-4bcc-bb51-4cac12e7a854 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.406077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecf4e0-5722-4bcc-bb51-4cac12e7a854 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.406077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ecf4e0-5722-4bcc-bb51-4cac12e7a854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.475996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c8ecf4e0-5722-4bcc-bb51-4cac12e7a854 None None] MFA Rules not processed for user `5ebda67a890d4a64a0eb12ec67c7d3df`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:42.477557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c8ecf4e0-5722-4bcc-bb51-4cac12e7a854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:42.546037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8754111d-49e6-4b0c-af8b-766ce6b0cdf4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:42.580925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 850/1716] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:42.582257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.582814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.583683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.622421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] MFA Rules not processed for user `af57181073784691ba5562c83b65beea`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:42.624122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:42.693119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:42.693119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.693637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.693838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.695446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.700750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:42.771628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 867/1718] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/projects => generated 387 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:42.771981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 851/1718] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:42.772060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:42.772128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:42.781463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609df615-ec98-4154-b93a-d8cb0189a070 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:42.883482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:42.906388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88abd47-abde-4301-967e-622762edf5c7 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:42.910811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:42.912280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:42.913095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.913784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.914134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.916180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.918979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:42.924902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.925444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.925970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.927447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.931170 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:42.946004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:42.950056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 868/1719] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/users => generated 309 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:42.951008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:42.988044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fdc3a0-4aad-4430-bef9-82611bc93b5e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:42.994219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b074b3-f764-408c-9f9e-02d47b513d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:42.994919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b074b3-f764-408c-9f9e-02d47b513d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:42.995196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b074b3-f764-408c-9f9e-02d47b513d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:42.995546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b074b3-f764-408c-9f9e-02d47b513d9a admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:42.996217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b074b3-f764-408c-9f9e-02d47b513d9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:42.998464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b074b3-f764-408c-9f9e-02d47b513d9a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.040628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 852/1720] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/projects => generated 409 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:43.041359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.046869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 869/1721] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:42 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:43.048022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c37bbe-18e6-42f0-b7e9-38c2aab45b7f None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.052439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c37bbe-18e6-42f0-b7e9-38c2aab45b7f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.052771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c37bbe-18e6-42f0-b7e9-38c2aab45b7f None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.054588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 870/1722] 10.222.0.43 () {54 vars in 960 bytes} [Fri Jun 24 08:41:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:43.055461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c37bbe-18e6-42f0-b7e9-38c2aab45b7f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.154179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe83fb04-2d63-4bf1-8211-c497b0795feb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.159979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba57360-a552-438f-898f-31240e901b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.160907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba57360-a552-438f-898f-31240e901b1a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.166710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba57360-a552-438f-898f-31240e901b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.167159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba57360-a552-438f-898f-31240e901b1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.168137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba57360-a552-438f-898f-31240e901b1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.173845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba57360-a552-438f-898f-31240e901b1a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.219276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c37bbe-18e6-42f0-b7e9-38c2aab45b7f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.220873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 853/1723] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:43 2022] POST /identity/v3/projects => generated 371 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:43.222805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.223323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.224551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.247741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] MFA Rules not processed for user `9abdce8004e24e81bd88bee757d8ec93`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:43.249161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:43.315711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.316455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.316756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.317013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.318461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.320612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.325100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 854/1724] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:43.328225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.336846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 871/1725] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:43.338795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.351089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb4df5a-1d8d-404b-a376-f3df41e2f2b6 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.356745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219d86f-0675-4d1f-8903-9246244c2030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.357709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340fb24f-4b56-4144-bdba-56ec24330278 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.357859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219d86f-0675-4d1f-8903-9246244c2030 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.357859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219d86f-0675-4d1f-8903-9246244c2030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.357995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219d86f-0675-4d1f-8903-9246244c2030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.359988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219d86f-0675-4d1f-8903-9246244c2030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.362129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219d86f-0675-4d1f-8903-9246244c2030 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.368708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.369409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.369690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.369968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.371125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.373815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.380839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 872/1726] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:43.380839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.380839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.380839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.409724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:43.415983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:43.434161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 855/1727] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:43 2022] POST /identity/v3/users => generated 311 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:43.434161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.463541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976b5e3b-21f7-44e8-bd4a-2c1564e14831 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.479943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.480644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.480924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.481198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.481894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.490591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.552443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 856/1728] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:43 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:43.553328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.573809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fe7219-28b8-4ede-acad-acb63bb88b22 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.581579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 873/1729] 10.222.0.43 () {58 vars in 1057 bytes} [Fri Jun 24 08:41:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:43.582449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.590198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:43.599138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.600076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.600364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.600712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] PATH_INFO: `/v3/projects/62caf9f4c9e94b348136c8d3634503f7/users/6ec7f0d2d74049c994644b93f01a696b/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.602594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3971ed-edc0-424a-b68f-5c39cef9dbfb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.609683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.611130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.611669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.612157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] PATH_INFO: `/v3/projects/2ed82032134b484b9f01812b128433fb/users/0c039e5d4b9446e3b695de8f2d6322bb/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.633164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=62caf9f4c9e94b348136c8d3634503f7, user_id=6ec7f0d2d74049c994644b93f01a696b, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.635418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.663547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 857/1730] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:43 2022] PUT /identity/v3/projects/62caf9f4c9e94b348136c8d3634503f7/users/6ec7f0d2d74049c994644b93f01a696b/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:43.664706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.697164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:43.716283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed82032134b484b9f01812b128433fb, user_id=0c039e5d4b9446e3b695de8f2d6322bb, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.734115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.774197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 874/1731] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:43 2022] PUT /identity/v3/projects/2ed82032134b484b9f01812b128433fb/users/0c039e5d4b9446e3b695de8f2d6322bb/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:43.778814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:43.857815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bead3-e846-4763-8126-39ce6a2abddf admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.945411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb13cd5-236f-4d4b-89b8-73918a224334 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:43.951027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:43.952089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:43.952587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:43.953092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:43.954760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:43.957370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:43.971448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 875/1732] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:43.972860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.061252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0efe064-03db-4a42-afe6-660944526739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.062785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efe064-03db-4a42-afe6-660944526739 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.063271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efe064-03db-4a42-afe6-660944526739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.063762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0efe064-03db-4a42-afe6-660944526739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.064912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efe064-03db-4a42-afe6-660944526739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.066950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0efe064-03db-4a42-afe6-660944526739 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.097081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 858/1733] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 433 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:44.103551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.104331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.104820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.134367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3e5d8-545a-48a6-9cff-8a09b0edc5a1 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.149867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.151789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.155121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.155721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.160105 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:44.164596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] MFA Rules not processed for user `12f0ce0a293f4114a90098985e5f21ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:44.167147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:44.190538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.192597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.223235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 876/1734] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:44.225807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.246797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc098615-3b17-4414-9757-e9039f981a3c service placement] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.261777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.262639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.263152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.263543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.264385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.266468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.293973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 877/1735] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:44.295064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.328280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7f980c-9471-4787-a319-17d872b724f1 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.339833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 859/1736] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:44.340868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.350793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.351686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.352064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.352463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.353296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.356927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.381065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09e383-1674-49d4-b40d-d0a79c2daeef None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.391317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 878/1737] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:44.393326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.397360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.397944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.400162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.403147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] PATH_INFO: `/v3/projects/f40f2a26410a4a69a9a941505a07694f/users/763b856a0dd24ad584bf322e6fc41f80/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.437017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e51e8-3a61-4f90-91c2-122498db579c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.455890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.456721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40f2a26410a4a69a9a941505a07694f, user_id=763b856a0dd24ad584bf322e6fc41f80, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.460794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.461136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.461541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.463495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] PATH_INFO: `/v3/projects/0355c0bf331f49dc9c3e9c997725232c/users/b36b0fdb1a3141418718d9875c84a0ae/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.500968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 860/1738] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:44 2022] PUT /identity/v3/projects/f40f2a26410a4a69a9a941505a07694f/users/763b856a0dd24ad584bf322e6fc41f80/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:44.501637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.509752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0355c0bf331f49dc9c3e9c997725232c, user_id=b36b0fdb1a3141418718d9875c84a0ae, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.518413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.547681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 879/1739] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:44 2022] PUT /identity/v3/projects/0355c0bf331f49dc9c3e9c997725232c/users/b36b0fdb1a3141418718d9875c84a0ae/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:44.548434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.562494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:44.615970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:44.676626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecdd429-bb8a-42fd-9899-c6631f057764 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.748647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9e985-9542-4be9-b412-adde83b9dbf0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:44.761325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.763395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.763705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.763971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] PATH_INFO: `/v3/projects/2ed82032134b484b9f01812b128433fb/users/0c039e5d4b9446e3b695de8f2d6322bb/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.771304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed82032134b484b9f01812b128433fb, user_id=0c039e5d4b9446e3b695de8f2d6322bb, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.776883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.801207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 880/1740] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:44 2022] PUT /identity/v3/projects/2ed82032134b484b9f01812b128433fb/users/0c039e5d4b9446e3b695de8f2d6322bb/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:44.812995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:44.815439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:44.881912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:44.884945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:44.886409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:44.886795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] PATH_INFO: `/v3/projects/62caf9f4c9e94b348136c8d3634503f7/users/6ec7f0d2d74049c994644b93f01a696b/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:44.894762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62caf9f4c9e94b348136c8d3634503f7, user_id=6ec7f0d2d74049c994644b93f01a696b, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:44.903856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:44.949367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 861/1741] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:44 2022] PUT /identity/v3/projects/62caf9f4c9e94b348136c8d3634503f7/users/6ec7f0d2d74049c994644b93f01a696b/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:44.962332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:45.070472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c8cb2e-18a6-4b3f-bd52-ef8ad9ce22fc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:45.257375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45df600-1bb8-4d47-ac75-4e74ecdb8711 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:45.265554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:45.266250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.266575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.266854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.268763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:45.271026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:45.276234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:45.277160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.282923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.283413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.284608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 862/1742] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:45.286283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:45.325092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dd3e4-437c-48ce-91f3-89d914032f81 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:45.331366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844d621-76e6-4157-806f-39c5a3235342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:45.333551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844d621-76e6-4157-806f-39c5a3235342 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.334006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844d621-76e6-4157-806f-39c5a3235342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.334434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844d621-76e6-4157-806f-39c5a3235342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.335490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844d621-76e6-4157-806f-39c5a3235342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:45.337316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844d621-76e6-4157-806f-39c5a3235342 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:45.383627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 863/1743] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:45.385215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac189e98-2b65-4f7e-bcce-b2e4fa5ffa9e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.385511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac189e98-2b65-4f7e-bcce-b2e4fa5ffa9e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.385796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac189e98-2b65-4f7e-bcce-b2e4fa5ffa9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.461045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:45.510157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:45.513740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:45.537984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ac189e98-2b65-4f7e-bcce-b2e4fa5ffa9e None None] MFA Rules not processed for user `0c039e5d4b9446e3b695de8f2d6322bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:45.541737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ac189e98-2b65-4f7e-bcce-b2e4fa5ffa9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:45.542250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 881/1744] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:45.545375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:45.572391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1a442-b75e-4ee0-8085-130e1d574e44 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:45.595470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:45.596403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.596684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.596961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] PATH_INFO: `/v3/projects/f40f2a26410a4a69a9a941505a07694f/users/763b856a0dd24ad584bf322e6fc41f80/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.602795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f40f2a26410a4a69a9a941505a07694f, user_id=763b856a0dd24ad584bf322e6fc41f80, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:45.605612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:45.650743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 882/1745] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:45 2022] PUT /identity/v3/projects/f40f2a26410a4a69a9a941505a07694f/users/763b856a0dd24ad584bf322e6fc41f80/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:45.662190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:45.725120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:45.841545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 864/1746] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:45 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 457 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:45.855425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafa2bb7-e943-414a-bad1-aad1a53558bb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:45.867544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39999261-2b8c-44ed-9cf7-bc67d010674c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.870640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39999261-2b8c-44ed-9cf7-bc67d010674c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.871743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39999261-2b8c-44ed-9cf7-bc67d010674c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.875917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:45.876206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:45.876327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:45.876651 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] PATH_INFO: `/v3/projects/0355c0bf331f49dc9c3e9c997725232c/users/b36b0fdb1a3141418718d9875c84a0ae/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:45.882527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0355c0bf331f49dc9c3e9c997725232c, user_id=b36b0fdb1a3141418718d9875c84a0ae, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:45.894060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:45.969930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 883/1747] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:45 2022] PUT /identity/v3/projects/0355c0bf331f49dc9c3e9c997725232c/users/b36b0fdb1a3141418718d9875c84a0ae/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:45.971703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:45.973548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:46.004323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-39999261-2b8c-44ed-9cf7-bc67d010674c None None] MFA Rules not processed for user `763b856a0dd24ad584bf322e6fc41f80`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:46.017226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-39999261-2b8c-44ed-9cf7-bc67d010674c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:46.170533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c688e-909c-42f9-87e6-7d67f9785f40 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:46.191557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3ef209-a439-4645-9688-86d4a31f6193 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:46.192629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef209-a439-4645-9688-86d4a31f6193 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.192980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef209-a439-4645-9688-86d4a31f6193 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:46.193338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3ef209-a439-4645-9688-86d4a31f6193 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.322463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 865/1748] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:46.323957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.325557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:46.325920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.401865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ef209-a439-4645-9688-86d4a31f6193 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:46.408699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3ef209-a439-4645-9688-86d4a31f6193 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:46.428940 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] MFA Rules not processed for user `b36b0fdb1a3141418718d9875c84a0ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:46.434879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:46.445573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 884/1749] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:45 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:46.450659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331da337-29e9-408b-a3d0-5be73b937c5d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.452105 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331da337-29e9-408b-a3d0-5be73b937c5d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:46.452957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331da337-29e9-408b-a3d0-5be73b937c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.541052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-331da337-29e9-408b-a3d0-5be73b937c5d None None] MFA Rules not processed for user `f83e72cd0d7e47b6bbfc4ffe64b71922`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:46.543190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-331da337-29e9-408b-a3d0-5be73b937c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:46.594304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:46.742905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 866/1750] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:46 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 417 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:46.743801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:46.749276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 885/1751] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:46.749659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:46.762924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.766474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:46.767038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.795974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] MFA Rules not processed for user `f83e72cd0d7e47b6bbfc4ffe64b71922`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:46.797368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70bf05c-7a23-41ae-9b51-4392f80d0970 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:46.799677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:46.810525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 886/1752] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:46.814779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:46.815494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.815783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:46.816063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.822987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:46.859489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:46.868339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:46.912186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 867/1753] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:46.914723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:46.956439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83625898-ed54-4e57-bfd5-e49f0c473f7b service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:46.971591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:46.975554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.975848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:46.976136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:46.992626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fe93d0-ad50-48ce-88ce-950d71cf5e5f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:46.998687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3e077-7809-476d-9385-75be9c92f01c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:46.999563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e077-7809-476d-9385-75be9c92f01c service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:46.999964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e077-7809-476d-9385-75be9c92f01c service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:47.000381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3e077-7809-476d-9385-75be9c92f01c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:47.015037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3e077-7809-476d-9385-75be9c92f01c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:47.019184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:47.020283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3e077-7809-476d-9385-75be9c92f01c service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:47.025796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:47.034863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 887/1754] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:47.086638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 868/1755] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:46 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:47.219774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:47.220484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:47.220975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:47.290813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] MFA Rules not processed for user `6ec7f0d2d74049c994644b93f01a696b`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:47.295141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:47.378068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:47.396876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d333475-b969-4ced-8ea0-a5b7e7b88cc4 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:47.409871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f5793-6063-4f11-b46c-98c5b3f30906 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:47.410554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f5793-6063-4f11-b46c-98c5b3f30906 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:47.411386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f5793-6063-4f11-b46c-98c5b3f30906 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:47.411691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f5793-6063-4f11-b46c-98c5b3f30906 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:47.455869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:47.568964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 888/1756] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:47.576207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f5793-6063-4f11-b46c-98c5b3f30906 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:47.578021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f5793-6063-4f11-b46c-98c5b3f30906 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:47.616650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 869/1757] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:47.662151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:47.678776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59583ab-94e6-4993-a92b-f6de8de9ef2e None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:47.691350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:47.692040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:47.692421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:47.692708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:47.719094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:47.770789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b077a5-903a-4181-8080-31273acda07a None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:47.771350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b077a5-903a-4181-8080-31273acda07a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:47.771825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b077a5-903a-4181-8080-31273acda07a None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:47.774784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 870/1758] 10.222.0.43 () {54 vars in 912 bytes} [Fri Jun 24 08:41:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:47.785045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:47.785916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:47.786977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:47.836480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:47.838726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:47.894767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:47.899439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:47.961790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 889/1759] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:47 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:48.072268 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 871/1760] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:41:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:48.097963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:48.114914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f46e45-5490-4ab1-aaa8-434ea0f544d6 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:48.141111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb00d6-a318-486c-9654-af5c7566ea45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:48.141874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb00d6-a318-486c-9654-af5c7566ea45 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:48.142161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb00d6-a318-486c-9654-af5c7566ea45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:48.142440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb00d6-a318-486c-9654-af5c7566ea45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:48.175868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb00d6-a318-486c-9654-af5c7566ea45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:48.185230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb00d6-a318-486c-9654-af5c7566ea45 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:48.233989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 890/1761] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:48.310342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:48.312028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:48.518343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b690c25b-5c36-4db5-847c-485d0741ce08 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:48.533200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:48.534260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:48.534390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:48.542344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] PATH_INFO: `/v3/users/826d10a724bc4c34bb8807220898b175` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:48.548372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] RBAC: Authorizing `identity:delete_user(user_id=826d10a724bc4c34bb8807220898b175)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:48.550379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:48.675126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '826d10a724bc4c34bb8807220898b175'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:48.696346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '826d10a724bc4c34bb8807220898b175'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:48.730944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:48.752436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '826d10a724bc4c34bb8807220898b175'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:48.777443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '826d10a724bc4c34bb8807220898b175'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:48.778696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '826d10a724bc4c34bb8807220898b175'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:48.798193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:48.798579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:48.798894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:48.800443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 872/1762] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:48 2022] DELETE /identity/v3/users/826d10a724bc4c34bb8807220898b175 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:48.850726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:48.852103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:49.062381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 891/1763] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:49.266370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:49.271895 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:49.318855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792f545-b05d-44e4-b326-dc9435fd0933 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:49.332664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:49.334670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:49.355530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:49.360756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:49.361057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:49.361328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] PATH_INFO: `/v3/users/89371fb7d9c34dca8a81c5dfe6d2acd6` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:49.374856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89371fb7d9c34dca8a81c5dfe6d2acd6)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:49.380644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:49.603113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89371fb7d9c34dca8a81c5dfe6d2acd6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:49.608867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7866a298-e0c4-4824-8080-5275d51244b4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:49.618517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ab967d-04d7-47d4-8107-8f8cd0b9445d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:49.621308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89371fb7d9c34dca8a81c5dfe6d2acd6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:49.623954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab967d-04d7-47d4-8107-8f8cd0b9445d service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:49.624287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab967d-04d7-47d4-8107-8f8cd0b9445d service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:49.627761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ab967d-04d7-47d4-8107-8f8cd0b9445d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:49.657362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab967d-04d7-47d4-8107-8f8cd0b9445d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:49.662463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ab967d-04d7-47d4-8107-8f8cd0b9445d service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:49.662918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89371fb7d9c34dca8a81c5dfe6d2acd6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:49.675999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89371fb7d9c34dca8a81c5dfe6d2acd6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:49.677493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff4013b4-82a5-490d-b6fc-22f5d4a69695 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89371fb7d9c34dca8a81c5dfe6d2acd6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:49.685376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 873/1764] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:49 2022] DELETE /identity/v3/users/89371fb7d9c34dca8a81c5dfe6d2acd6 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:49.686827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:49.687246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:49.687703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:49.690205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 892/1765] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:49.695536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:49.695820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:49.696094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:49.717032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:49.725038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:49.738257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:49.743788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:49.841945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:49.842482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 874/1766] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:41:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:49.915583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 893/1767] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:49.985511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:49.987852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:50.132101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:50.133803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:50.140140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da60a154-aa65-41a9-a855-b8586d1a1b54 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:50.146005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b8166-49e4-4243-bde1-6b27cffb4eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:50.146953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b8166-49e4-4243-bde1-6b27cffb4eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:50.147510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b8166-49e4-4243-bde1-6b27cffb4eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:50.148278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b8166-49e4-4243-bde1-6b27cffb4eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:50.171694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b8166-49e4-4243-bde1-6b27cffb4eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:50.176360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b8166-49e4-4243-bde1-6b27cffb4eb0 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:50.200386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 875/1768] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:50.307702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aa1e74-1f87-4f5b-bc3f-1764d8f8b932 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:50.316500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8223c2-06cc-4530-8b2d-51ed5fe2d827 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:50.317908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8223c2-06cc-4530-8b2d-51ed5fe2d827 service swift] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:50.318337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8223c2-06cc-4530-8b2d-51ed5fe2d827 service swift] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:50.318748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8223c2-06cc-4530-8b2d-51ed5fe2d827 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:50.340073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8223c2-06cc-4530-8b2d-51ed5fe2d827 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:50.342372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8223c2-06cc-4530-8b2d-51ed5fe2d827 service swift] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:50.359615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 894/1769] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:41:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:41:51.667577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9c52d-ada2-4cfe-aed4-05bbd2397cfb None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:51.668112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9c52d-ada2-4cfe-aed4-05bbd2397cfb None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:51.668321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9c52d-ada2-4cfe-aed4-05bbd2397cfb None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:51.672166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 876/1770] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:41:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:41:51.927372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:51.927914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:51.927914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:51.959246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:51.960533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:52.189624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 895/1771] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:41:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:52.349179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9c52d-ada2-4cfe-aed4-05bbd2397cfb None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:52.351090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-eed9c52d-ada2-4cfe-aed4-05bbd2397cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:52.380673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:52.399143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9c52d-ada2-4cfe-aed4-05bbd2397cfb None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:52.451359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:52.452058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:52.452330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:52.452606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:52.482756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:52.484814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:52.528974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 877/1772] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:52.898551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:52.900405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:52.925712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ad38d-8233-43b0-9260-883c29ee7a60 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:52.939468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:52.946156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:52.946950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:52.947655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] PATH_INFO: `/v3/projects/465ad3d0ecd242d58f347fff5771468f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:52.952185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=465ad3d0ecd242d58f347fff5771468f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:52.967440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:52.969451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:52.970714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:52.993247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f687-0019-41f6-a320-dfb7d521479d service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:52.999079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:53.000308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.000757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.001176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] PATH_INFO: `/v3/projects/0586fe42a68745ec86eaca054e20ffea` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.003394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0586fe42a68745ec86eaca054e20ffea)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:53.006631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:53.108169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '465ad3d0ecd242d58f347fff5771468f', 'request_id': 'req-49301b46-f937-48aa-8181-9ca0d01b1241'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.127523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '465ad3d0ecd242d58f347fff5771468f', 'request_id': 'req-49301b46-f937-48aa-8181-9ca0d01b1241'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.143619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '465ad3d0ecd242d58f347fff5771468f', 'request_id': 'req-49301b46-f937-48aa-8181-9ca0d01b1241'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.149340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:53.151471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0586fe42a68745ec86eaca054e20ffea', 'request_id': 'req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.181179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0586fe42a68745ec86eaca054e20ffea', 'request_id': 'req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.192784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] The token cache is being invalidate because project 465ad3d0ecd242d58f347fff5771468f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:53.193004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.193892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.199365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0586fe42a68745ec86eaca054e20ffea', 'request_id': 'req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.206173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 896/1773] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:52 2022] DELETE /identity/v3/projects/465ad3d0ecd242d58f347fff5771468f => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:53.211965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:53.223408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:53.230600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] The token cache is being invalidate because project 0586fe42a68745ec86eaca054e20ffea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:53.230923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.231967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ea7e5aa7-3bba-48d2-8d61-407e3d59624f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.234580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 878/1774] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:52 2022] DELETE /identity/v3/projects/0586fe42a68745ec86eaca054e20ffea => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:53.241316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64094635-68c5-4b8d-92aa-fac3d51d8463 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.242378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64094635-68c5-4b8d-92aa-fac3d51d8463 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.242960 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64094635-68c5-4b8d-92aa-fac3d51d8463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.289910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-64094635-68c5-4b8d-92aa-fac3d51d8463 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:53.291741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-64094635-68c5-4b8d-92aa-fac3d51d8463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:53.363202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301b46-f937-48aa-8181-9ca0d01b1241 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:53.440776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:53.442147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.442612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.443103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] PATH_INFO: `/v3/users/0c039e5d4b9446e3b695de8f2d6322bb` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.444670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c039e5d4b9446e3b695de8f2d6322bb)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:53.447183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:53.457745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 879/1775] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:53.461018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfddc9-d684-4c52-a63c-e24b9e2d59f8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.461544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfddc9-d684-4c52-a63c-e24b9e2d59f8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.462026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cfddc9-d684-4c52-a63c-e24b9e2d59f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.494702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-99cfddc9-d684-4c52-a63c-e24b9e2d59f8 None None] MFA Rules not processed for user `12f0ce0a293f4114a90098985e5f21ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:53.496024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-99cfddc9-d684-4c52-a63c-e24b9e2d59f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:53.513440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c039e5d4b9446e3b695de8f2d6322bb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.524110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c039e5d4b9446e3b695de8f2d6322bb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.536312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c039e5d4b9446e3b695de8f2d6322bb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.537436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c039e5d4b9446e3b695de8f2d6322bb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.538481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c039e5d4b9446e3b695de8f2d6322bb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:53.566842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 880/1776] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:53.568254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.568586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.568865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.580498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 897/1777] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:53 2022] DELETE /identity/v3/users/0c039e5d4b9446e3b695de8f2d6322bb => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:53.613237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:53.614778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:53.632381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:53.705948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 881/1778] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:53.708052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:53.728217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1b0ca0-1bf7-43c9-84a2-96bf719fbc01 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:53.744612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:53.745542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.746061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.746547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.747544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:53.748308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:53.750690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:53.772155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 898/1779] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:41:53.776251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:53.796926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f8edf3-9b3e-42da-bbc3-b95c56c43d43 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:53.806601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:53.807859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:53.808372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:53.809222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:53.811443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:53.814912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:53.867845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 882/1780] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:53 2022] POST /identity/v3/projects => generated 365 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:53.871895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:53.882229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:53.889850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29947a-b1da-4934-8b35-edf99cb9bb95 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.066131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.066846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.067712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.067938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.068657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.070767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.117770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 899/1781] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:53 2022] POST /identity/v3/users => generated 283 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:54.122303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.136513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64e7c1-9341-45c7-b336-62121b89d15a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.152198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82653100-427c-4313-a375-e008f594394f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.153198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82653100-427c-4313-a375-e008f594394f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.153705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82653100-427c-4313-a375-e008f594394f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.154579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82653100-427c-4313-a375-e008f594394f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.155606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82653100-427c-4313-a375-e008f594394f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.157879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82653100-427c-4313-a375-e008f594394f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.187704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 883/1782] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:54.190929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.205808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82653100-427c-4313-a375-e008f594394f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.222151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82383ab8-6546-4eff-aa89-54777a69cacd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.224785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-82653100-427c-4313-a375-e008f594394f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.256423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.261582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.268498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.271334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.272323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.274142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.298003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82653100-427c-4313-a375-e008f594394f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.330372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.331305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.331584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.334856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] PATH_INFO: `/v3/projects/2ed82032134b484b9f01812b128433fb` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.338390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 900/1783] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:54 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:54.339356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed82032134b484b9f01812b128433fb)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.343177 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.344316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.383052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a549df-4da4-4b36-874f-c8526e03c8b4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.397536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ce78e4-261b-416d-bb32-68490b1f87d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.400582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce78e4-261b-416d-bb32-68490b1f87d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.401186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce78e4-261b-416d-bb32-68490b1f87d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.401707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ce78e4-261b-416d-bb32-68490b1f87d3 admin admin] PATH_INFO: `/v3/projects/36fd432b4335442990c31d60931f1b8b/users/2a58579bdb2f453591bac996511af7f5/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.444811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce78e4-261b-416d-bb32-68490b1f87d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36fd432b4335442990c31d60931f1b8b, user_id=2a58579bdb2f453591bac996511af7f5, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.457303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ce78e4-261b-416d-bb32-68490b1f87d3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.477062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed82032134b484b9f01812b128433fb', 'request_id': 'req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.493248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 901/1784] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:54 2022] PUT /identity/v3/projects/36fd432b4335442990c31d60931f1b8b/users/2a58579bdb2f453591bac996511af7f5/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:54.496947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.497282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.497645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.506665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed82032134b484b9f01812b128433fb', 'request_id': 'req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.539027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed82032134b484b9f01812b128433fb', 'request_id': 'req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.570379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] MFA Rules not processed for user `2a58579bdb2f453591bac996511af7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:54.571682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.585191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] The token cache is being invalidate because project 2ed82032134b484b9f01812b128433fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:54.585470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.586233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-7a97e8cf-15ef-45ce-9273-7a39c7908ae7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:54.588018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 884/1785] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:54 2022] DELETE /identity/v3/projects/2ed82032134b484b9f01812b128433fb => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:54.603611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e329e-db81-40d3-9108-ea64fcea7082 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.604028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e329e-db81-40d3-9108-ea64fcea7082 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.604378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8e329e-db81-40d3-9108-ea64fcea7082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.646923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6d8e329e-db81-40d3-9108-ea64fcea7082 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:54.648879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6d8e329e-db81-40d3-9108-ea64fcea7082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.722860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 902/1786] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:54.737264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.742903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.867629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 885/1787] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:54.872958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.873333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.873685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.902038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:54.907729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:54.910374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.913481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be2022-774c-4a1e-9799-6c624cee2864 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.918706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.919507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.919865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.920219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.931114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 886/1788] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:54.931114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.933113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:54.939228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.940945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.946902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:54.963101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 903/1789] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:54.964103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:54.979850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bd3765-efd1-472e-b380-0eed1c848a97 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.981284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7566a4-fd4a-4d1d-b0de-e90728bccd57 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:54.988501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.989888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.990017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:54.990204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.990573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.990958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:54.991262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:54.991794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] PATH_INFO: `/v3/users/d513f970536a4876b16d3e122967dbd4` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:54.992404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:54.993024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.996142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d513f970536a4876b16d3e122967dbd4)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:54.997045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:54.998146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.026145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 904/1790] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:55.027395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.059530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e277567e-fbb6-4838-b3ea-b69af069fa3d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.072690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.073505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.073656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.073982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.074689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.077787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.104924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd513f970536a4876b16d3e122967dbd4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.126929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd513f970536a4876b16d3e122967dbd4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.145167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 905/1791] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:55 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:55.151188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.169243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd513f970536a4876b16d3e122967dbd4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.193306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd513f970536a4876b16d3e122967dbd4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.194835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd513f970536a4876b16d3e122967dbd4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:55.213787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 887/1792] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:54 2022] DELETE /identity/v3/users/d513f970536a4876b16d3e122967dbd4 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:55.283478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6f59d-67fc-483e-b0e6-6fb7347b492b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.402047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.402835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.403235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.403602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.404415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.407923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.451875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 906/1793] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:55 2022] POST /identity/v3/users => generated 290 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:55.452596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.454270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:55.475193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84708a6-a1bd-4acc-8ca5-756cf451f200 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.487370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.488657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.489272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.489748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.490727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.492659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.510618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 888/1794] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:55.515719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.530346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9abf5b7-c34f-43f3-9a38-abc8fefda5fe admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.539702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef4695-14c9-45e5-a3b8-92fd5ebb24ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.540840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef4695-14c9-45e5-a3b8-92fd5ebb24ec admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.541281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef4695-14c9-45e5-a3b8-92fd5ebb24ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.541593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef4695-14c9-45e5-a3b8-92fd5ebb24ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.543317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef4695-14c9-45e5-a3b8-92fd5ebb24ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.552820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef4695-14c9-45e5-a3b8-92fd5ebb24ec admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.571284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.583144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 907/1795] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:55 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:55.584936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ca40b-58ce-4b66-a6b1-a7a9f330225f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.591560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.593031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.593545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.594087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] PATH_INFO: `/v3/projects/6546d8b2a3c146818d8fddbf739d2347/users/1742c72f76a84e67a9cec199fbb1b298/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.619858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6546d8b2a3c146818d8fddbf739d2347, user_id=1742c72f76a84e67a9cec199fbb1b298, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.623511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:55.673819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 889/1796] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:55 2022] PUT /identity/v3/projects/6546d8b2a3c146818d8fddbf739d2347/users/1742c72f76a84e67a9cec199fbb1b298/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:55.676041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.676340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.677333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.762937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.766338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:55.813560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] MFA Rules not processed for user `1742c72f76a84e67a9cec199fbb1b298`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:55.814949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:55.872690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:55.970654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd024ae-a242-409a-b6c0-e127aef3629e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:55.975032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 908/1797] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:55.979257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:55.980477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:55.980992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:55.986994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] PATH_INFO: `/v3/projects/78a138fb930741858ff86e0322818521` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:55.990145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:55.991051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a138fb930741858ff86e0322818521)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:55.993196 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.019401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527dcf1-3229-4bcb-8e58-a5d325a4705a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.026879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9003281c-ba97-44d7-b203-943dfe4a7d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.027845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003281c-ba97-44d7-b203-943dfe4a7d57 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.028360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003281c-ba97-44d7-b203-943dfe4a7d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.028825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9003281c-ba97-44d7-b203-943dfe4a7d57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.029681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003281c-ba97-44d7-b203-943dfe4a7d57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.032157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9003281c-ba97-44d7-b203-943dfe4a7d57 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.124937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 909/1798] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:55 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:56.127341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:56.128703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.129003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.129310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.148689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a138fb930741858ff86e0322818521', 'request_id': 'req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.160074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a138fb930741858ff86e0322818521', 'request_id': 'req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.179516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a138fb930741858ff86e0322818521', 'request_id': 'req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.195364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] The token cache is being invalidate because project 78a138fb930741858ff86e0322818521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:56.195849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.197053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.199265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 890/1799] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:55 2022] DELETE /identity/v3/projects/78a138fb930741858ff86e0322818521 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:56.200187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.202965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.232677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:56.240052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.296049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b9cc6-6fd0-4081-8c25-c4273f36f561 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.411266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.412650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.413088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.413562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.414599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.417685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.462708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 891/1800] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:56.472568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.472568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.473102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 910/1801] 10.222.0.46 () {58 vars in 1015 bytes} [Fri Jun 24 08:41:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:56.473102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.487778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c03d604-e227-4337-afdc-7448463e5d32 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.516119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d59017-1e07-4545-aa4c-c3888f6ea4e0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.520508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.522242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.522868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.523654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] PATH_INFO: `/v3/users/6ec7f0d2d74049c994644b93f01a696b` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.526236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.526792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ec7f0d2d74049c994644b93f01a696b)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.529821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.531097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.536524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.536981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.539786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.541694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.611014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 911/1802] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:56.612382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.642422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ec7f0d2d74049c994644b93f01a696b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.646120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d1135-52bd-4aac-8cec-5cc8a18407f4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.657348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ec7f0d2d74049c994644b93f01a696b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.666434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.667585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.667870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.668145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] PATH_INFO: `/v3/projects/3b16810da9f746e4b59333ee949623a7/users/d3f1885767fa4e6184e6aa525fadbb79/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.669656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=82,keystone:DELETE=13,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:56.685211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ec7f0d2d74049c994644b93f01a696b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.713596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ec7f0d2d74049c994644b93f01a696b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.714415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ec7f0d2d74049c994644b93f01a696b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:56.724541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 892/1803] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:56 2022] DELETE /identity/v3/users/6ec7f0d2d74049c994644b93f01a696b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:56.744848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b16810da9f746e4b59333ee949623a7, user_id=d3f1885767fa4e6184e6aa525fadbb79, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.757061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.793232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 912/1804] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:56 2022] PUT /identity/v3/projects/3b16810da9f746e4b59333ee949623a7/users/d3f1885767fa4e6184e6aa525fadbb79/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:56.797010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:56.799138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:56.897713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2ab7df-95d8-4555-bfcf-47bf6cbdc6d5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:56.964730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd1263-e6d9-4b19-b04c-f4b579768215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:56.964901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd1263-e6d9-4b19-b04c-f4b579768215 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:56.965134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd1263-e6d9-4b19-b04c-f4b579768215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:56.965415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd1263-e6d9-4b19-b04c-f4b579768215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:56.966160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd1263-e6d9-4b19-b04c-f4b579768215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:56.967846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd1263-e6d9-4b19-b04c-f4b579768215 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:56.981257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 893/1805] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:56 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:56.985549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.001385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31179b3e-f6c2-463d-a1a6-700e0d85c98a admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.007493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.008750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.008750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.009109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] PATH_INFO: `/v3/projects/3b16810da9f746e4b59333ee949623a7/users/d3f1885767fa4e6184e6aa525fadbb79/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.012062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b16810da9f746e4b59333ee949623a7, user_id=d3f1885767fa4e6184e6aa525fadbb79, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.014762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.028148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 913/1806] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:56 2022] PUT /identity/v3/projects/3b16810da9f746e4b59333ee949623a7/users/d3f1885767fa4e6184e6aa525fadbb79/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:57.039996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.040677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.041062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.078402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] MFA Rules not processed for user `d3f1885767fa4e6184e6aa525fadbb79`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:57.080145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.106124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.107859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.161745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 894/1807] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:57.172948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.174375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.184424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:57.185858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31633a0-cd61-4896-869d-90de77e57641 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.191194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.192110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.192943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.193217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] PATH_INFO: `/v3/projects/62caf9f4c9e94b348136c8d3634503f7` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.194443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62caf9f4c9e94b348136c8d3634503f7)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.196416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.208363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:57.234654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27330884-597c-4833-b4f6-3a26fb42e8f3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.246802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62caf9f4c9e94b348136c8d3634503f7', 'request_id': 'req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.253485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62caf9f4c9e94b348136c8d3634503f7', 'request_id': 'req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.259989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62caf9f4c9e94b348136c8d3634503f7', 'request_id': 'req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.273228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] The token cache is being invalidate because project 62caf9f4c9e94b348136c8d3634503f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:57.273548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.274199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5d7013b5-d2bb-4ccc-8005-1b259a299fe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.276061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 914/1808] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:57 2022] DELETE /identity/v3/projects/62caf9f4c9e94b348136c8d3634503f7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:57.280726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547cdd-cbf1-4153-b8d2-fadec599723b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.281015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547cdd-cbf1-4153-b8d2-fadec599723b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.281296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547cdd-cbf1-4153-b8d2-fadec599723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.291688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.292428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.292744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.293158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.295157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.300866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-98547cdd-cbf1-4153-b8d2-fadec599723b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:57.302102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-98547cdd-cbf1-4153-b8d2-fadec599723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.359167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.360580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.363693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 915/1809] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:57.370717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 895/1810] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:57.372829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.373128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.373410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.392137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:57.393494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.405055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 916/1811] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:57.410068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.421408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8da4b-ae2c-4dd3-8cb2-793e45f1b69c service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.426173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.427409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.427699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.427983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.428713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:57.429022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.433420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.445175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 896/1812] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:57.448639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.458023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d3290c-c0d3-4f74-8943-5785a7e64814 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.462686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.463400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.463667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.463939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.464634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.466765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.497590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 917/1813] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:57.501557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.517112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06830572-aaa4-406e-aa31-0f6f3089c126 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.616959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.617651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.617966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.618214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.618926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.624147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.629557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:57.644901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 897/1814] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:57.649646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.668536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944f4f8-4513-475b-85c8-7cb0f7fe2f7f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.674866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.675675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.676031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.676385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.677737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.679498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.696305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 918/1815] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:57.700281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.710455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9418bf5-bea9-4cb6-98a9-deb7500929f8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.719846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.720637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.720933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.721212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.721950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.724351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.735883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 898/1816] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:57.740657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.751842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9cf32f-fe88-4e0b-a66c-115bf16d60ff admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.762623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d974c40-336f-472c-a659-f7bf9e6731df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.763559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974c40-336f-472c-a659-f7bf9e6731df admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.763845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974c40-336f-472c-a659-f7bf9e6731df admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.764145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d974c40-336f-472c-a659-f7bf9e6731df admin admin] PATH_INFO: `/v3/projects/4d3f6de82dc74d38a6ca406bd320f448/users/813050e148ca4e6b8eb7e52a363736df/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.777846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.779699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.793776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f4e36f-ca8d-47ed-8e30-9a4d51b3087c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:57.798353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974c40-336f-472c-a659-f7bf9e6731df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d3f6de82dc74d38a6ca406bd320f448, user_id=813050e148ca4e6b8eb7e52a363736df, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.801502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d974c40-336f-472c-a659-f7bf9e6731df admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.808471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:57.809394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.809701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.810009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] PATH_INFO: `/v3/users/1742c72f76a84e67a9cec199fbb1b298` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.811597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1742c72f76a84e67a9cec199fbb1b298)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:57.813676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:57.818588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 919/1817] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:57 2022] PUT /identity/v3/projects/4d3f6de82dc74d38a6ca406bd320f448/users/813050e148ca4e6b8eb7e52a363736df/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:57.828632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.829402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.829402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.858379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] MFA Rules not processed for user `813050e148ca4e6b8eb7e52a363736df`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:57.859768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.871044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1742c72f76a84e67a9cec199fbb1b298'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.878551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1742c72f76a84e67a9cec199fbb1b298'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.886629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:57.891591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1742c72f76a84e67a9cec199fbb1b298'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.899906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1742c72f76a84e67a9cec199fbb1b298'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.900624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0af25b0-325c-45e5-916c-f4eb3de53014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1742c72f76a84e67a9cec199fbb1b298'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:57.906695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 899/1818] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:57 2022] DELETE /identity/v3/users/1742c72f76a84e67a9cec199fbb1b298 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:57.911247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:57.911535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:57.911822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:57.928945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:57.932475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 920/1819] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:57.932606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.933335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:57.934663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:57.992433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 900/1820] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:58.002152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.003443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975696b-16bb-4ebc-a33f-8829773cbcf5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.003563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.009059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.009733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.010016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.010298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.011052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.013071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.042641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 921/1821] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:57 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:58.043697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.065612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef931b2e-783d-4420-9654-f27b24b49d44 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.069328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.113725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef488760-a773-4520-a95d-cfa57093256c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.119172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.119957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.120354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.120635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.121327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.122786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.123253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.123646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.124001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.124354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.135733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.137682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.146165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 922/1822] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:58 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:58.150355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 901/1823] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:41:58.150563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.163342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c132e23b-709d-446a-b061-30b952c7b24f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.168219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.169341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.169855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.170366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.171244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.172975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.184008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 923/1824] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:58.188838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.200987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b31a9-ce07-4131-a054-834b503f4d9c service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.205861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6aca9-002e-4cd6-ba79-3a8fac59998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.207374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6aca9-002e-4cd6-ba79-3a8fac59998f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.207374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6aca9-002e-4cd6-ba79-3a8fac59998f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.207374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6aca9-002e-4cd6-ba79-3a8fac59998f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.207723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6aca9-002e-4cd6-ba79-3a8fac59998f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.209911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6aca9-002e-4cd6-ba79-3a8fac59998f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.220726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 902/1825] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:58.225189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.236524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea85ca-201f-4891-b311-0f4810c20868 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.241584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.242830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.242830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.243090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] PATH_INFO: `/v3/projects/65d16706dfba48c3a30362f9027a6df5/users/c495c2611f29499e9b3f967b8c09ee55/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.259050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65d16706dfba48c3a30362f9027a6df5, user_id=c495c2611f29499e9b3f967b8c09ee55, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.261915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.276298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 924/1826] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:58 2022] PUT /identity/v3/projects/65d16706dfba48c3a30362f9027a6df5/users/c495c2611f29499e9b3f967b8c09ee55/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:58.286950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.287286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.287535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.315049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] MFA Rules not processed for user `c495c2611f29499e9b3f967b8c09ee55`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:58.316315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.381165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 903/1827] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:58.385354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.394672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.435968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.437435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.464406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdadc0f-ef16-48a9-9a62-fc1b5326ea28 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.470255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.470916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.471263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.471543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.472240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.474312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.499924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 925/1828] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:58 2022] POST /identity/v3/projects => generated 375 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:41:58.504126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.508788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e7aa98-3a7a-4847-b379-39927b0da7f5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.541400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.569111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b2fb65-9835-49de-89ef-492e168970c9 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.573334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.573755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.574256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.574494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.574600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.574909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.575013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] PATH_INFO: `/v3/projects/6546d8b2a3c146818d8fddbf739d2347` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.575192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.575918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.576263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6546d8b2a3c146818d8fddbf739d2347)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.577825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.578293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.598372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 926/1829] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:41:58 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:41:58.603268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.618672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0cbf-5a4e-4211-aaee-49a03d3a5f9f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.624804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.625723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.626033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.626244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.627153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.629038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.641600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6546d8b2a3c146818d8fddbf739d2347', 'request_id': 'req-70e4577d-c3dd-484e-8464-5b586cbd30bc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.641759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 927/1830] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:41:58.646041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.649397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6546d8b2a3c146818d8fddbf739d2347', 'request_id': 'req-70e4577d-c3dd-484e-8464-5b586cbd30bc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.657059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d1d75-772f-4d83-b557-ad41a9082403 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.657541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6546d8b2a3c146818d8fddbf739d2347', 'request_id': 'req-70e4577d-c3dd-484e-8464-5b586cbd30bc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.673719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] The token cache is being invalidate because project 6546d8b2a3c146818d8fddbf739d2347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:58.674072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.678234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:58.679814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 904/1831] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:58 2022] DELETE /identity/v3/projects/6546d8b2a3c146818d8fddbf739d2347 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:58.683558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.685377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.726687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.727623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.727907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.728184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] PATH_INFO: `/v3/projects/0141955f05564a55acea7197ed67dd89/users/643b3eb172ed4f40bb856d4f37b24c3b/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.754757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0141955f05564a55acea7197ed67dd89, user_id=643b3eb172ed4f40bb856d4f37b24c3b, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.758176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.761920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.784803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 928/1832] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:58 2022] PUT /identity/v3/projects/0141955f05564a55acea7197ed67dd89/users/643b3eb172ed4f40bb856d4f37b24c3b/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:58.792610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.795369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:58.797081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4577d-c3dd-484e-8464-5b586cbd30bc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.894129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516763b-998d-4b83-8e30-015d1d115e85 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.896356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.899281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.899281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.899281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] PATH_INFO: `/v3/users/d3f1885767fa4e6184e6aa525fadbb79` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.900113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f1885767fa4e6184e6aa525fadbb79)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.902080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.918343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.919581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.921160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.921814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:58.922774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:58.925193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:58.939922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:58.950972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 929/1833] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:41:58 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:41:58.952656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:58.969522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c4b8b-3f00-4ab2-a302-d3e156afae10 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:58.992879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d0d3a1-94f7-4397-b85a-81bcc9568f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:58.995594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0d3a1-94f7-4397-b85a-81bcc9568f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:58.996760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0d3a1-94f7-4397-b85a-81bcc9568f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:58.997921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d0d3a1-94f7-4397-b85a-81bcc9568f70 admin admin] PATH_INFO: `/v3/projects/0141955f05564a55acea7197ed67dd89/users/643b3eb172ed4f40bb856d4f37b24c3b/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.002318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0d3a1-94f7-4397-b85a-81bcc9568f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0141955f05564a55acea7197ed67dd89, user_id=643b3eb172ed4f40bb856d4f37b24c3b, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.006795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d0d3a1-94f7-4397-b85a-81bcc9568f70 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.019353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 930/1834] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:41:58 2022] PUT /identity/v3/projects/0141955f05564a55acea7197ed67dd89/users/643b3eb172ed4f40bb856d4f37b24c3b/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:59.020600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f1885767fa4e6184e6aa525fadbb79'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.031153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.031425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.032663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.045179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f1885767fa4e6184e6aa525fadbb79'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.075826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f1885767fa4e6184e6aa525fadbb79'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.086218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f1885767fa4e6184e6aa525fadbb79'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.087034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f1885767fa4e6184e6aa525fadbb79'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.093205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] MFA Rules not processed for user `643b3eb172ed4f40bb856d4f37b24c3b`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:59.093684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 905/1835] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:41:58 2022] DELETE /identity/v3/users/d3f1885767fa4e6184e6aa525fadbb79 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:41:59.095135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.252659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 931/1836] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:41:59.265364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.267162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.353897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57805b-58c2-4ba5-afee-7a738280aca0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.358349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.368482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.368794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.369063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.379205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.380760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.391866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 906/1837] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:59.446477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.448432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.462620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9f2a85-3365-4970-bc46-916ab864062c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.467054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.467746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.468035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.468318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.470187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.500425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.502015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.572918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.574608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2654b4c-d4ca-4af1-ad13-8739aeaeccf8 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.579945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:59.588663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 932/1838] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:41:59.596896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94644c5-1547-4a8c-b8a3-b5cd3f1d30fc service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.603635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.604509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.605010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.605242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] PATH_INFO: `/v3/projects/3b16810da9f746e4b59333ee949623a7` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.606471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b16810da9f746e4b59333ee949623a7)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.608645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.653626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:41:59.666759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b16810da9f746e4b59333ee949623a7', 'request_id': 'req-144f7fcf-1d2f-4518-a229-93765e1bfbff'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.674746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b16810da9f746e4b59333ee949623a7', 'request_id': 'req-144f7fcf-1d2f-4518-a229-93765e1bfbff'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.681422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b16810da9f746e4b59333ee949623a7', 'request_id': 'req-144f7fcf-1d2f-4518-a229-93765e1bfbff'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.691924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] The token cache is being invalidate because project 3b16810da9f746e4b59333ee949623a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:41:59.692740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.693416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-144f7fcf-1d2f-4518-a229-93765e1bfbff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:41:59.695038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 907/1839] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:41:59 2022] DELETE /identity/v3/projects/3b16810da9f746e4b59333ee949623a7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:41:59.699966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.700193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.700549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.718902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:59.720433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.806470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 933/1840] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:59.814298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.815099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.815099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.852224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:41:59.853906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:41:59.863597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 908/1841] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:41:59.868229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.879366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a265add-164f-4e07-80b1-2a741fa92970 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.885463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.886298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.886584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.886857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.887708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:41:59.888113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.889719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:41:59.907444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 934/1842] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:41:59.908245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:41:59.918308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60666a79-66ec-4fa4-8b20-9b99d1625d11 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:41:59.943889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:41:59.944838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:41:59.945157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:41:59.945601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:41:59.946500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:41:59.949040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.021605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 909/1843] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:41:59 2022] POST /identity/v3/projects => generated 371 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:00.025842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.037271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cfe5c7-231f-4fa6-b486-793a8c2a9e13 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.110846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.111580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.111868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.112144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.112832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.115309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.141230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 935/1844] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:00 2022] POST /identity/v3/users => generated 286 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:00.142488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.158988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915817b6-fc5d-4e62-83aa-fc82bd0a9604 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.163882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.164734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.165410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.165831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.166554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.168184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.182717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 910/1845] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:00.183979 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.194610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc4b037-f7c6-480f-8d98-712d4aed64fe admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.199632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d5d29-918d-4813-8978-8cf284c2640a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.200472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d5d29-918d-4813-8978-8cf284c2640a admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.200615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d5d29-918d-4813-8978-8cf284c2640a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.200894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d5d29-918d-4813-8978-8cf284c2640a admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.201621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d5d29-918d-4813-8978-8cf284c2640a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.203204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d5d29-918d-4813-8978-8cf284c2640a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.213879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 936/1846] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:00.217973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.227836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8764685-8625-49f3-8d31-b688d65818fc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.232525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.233404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.233689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.233966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] PATH_INFO: `/v3/projects/3fd1e42945794a1096d0269bb1656482/users/40dac9dfb79d45d2a84d581698e5f663/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.249383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fd1e42945794a1096d0269bb1656482, user_id=40dac9dfb79d45d2a84d581698e5f663, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.252333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.265730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 911/1847] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:00 2022] PUT /identity/v3/projects/3fd1e42945794a1096d0269bb1656482/users/40dac9dfb79d45d2a84d581698e5f663/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:00.270615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91272a8-ccbb-4045-98e2-6dd288c28f73 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.271071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91272a8-ccbb-4045-98e2-6dd288c28f73 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.271167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91272a8-ccbb-4045-98e2-6dd288c28f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.288521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d91272a8-ccbb-4045-98e2-6dd288c28f73 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:00.289712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d91272a8-ccbb-4045-98e2-6dd288c28f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:00.346992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 937/1848] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:00.356623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:00.358269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:00.416718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b625c78d-9f35-4156-b581-dc4b5305fde9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:00.420463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:00.421157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:00.421438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:00.421753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:00.431480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:00.432914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:00.443064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 912/1849] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:01.241564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.241907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.242137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.255374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:01.276972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] MFA Rules not processed for user `40dac9dfb79d45d2a84d581698e5f663`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:01.279938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:01.425188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 938/1850] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:01.430649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.443437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377fb2b-a65c-4039-b4dc-e117a904152b service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.460776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.461472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.461768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.462046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.463016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.465155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.510213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 913/1851] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:01 2022] POST /identity/v3/projects => generated 371 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:01.517857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.565398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61739654-1d35-48c5-9cb8-574c393ea5aa None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.660719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.661608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.661970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.662256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.662956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.664920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.711442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 939/1852] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:01 2022] POST /identity/v3/users => generated 292 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:01.712568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.724696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a22937-c0d0-4b35-8f0d-a9f72346f9b2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.730989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.732023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.732658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.733150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.734232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.736149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.749567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 914/1853] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:01.753878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.765009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212273e2-2b0b-4770-b1cb-7b6dc1ff8faf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.769639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.770688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.770893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.771172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] PATH_INFO: `/v3/projects/4b45cba40d96425ab7e1cca987fdeb71/users/0311fbfbf7d64b1e9eebffa3f3b9600d/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.787870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b45cba40d96425ab7e1cca987fdeb71, user_id=0311fbfbf7d64b1e9eebffa3f3b9600d, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.790861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.808481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 940/1854] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:01 2022] PUT /identity/v3/projects/4b45cba40d96425ab7e1cca987fdeb71/users/0311fbfbf7d64b1e9eebffa3f3b9600d/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:01.812736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.823582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b01d9c7-9537-4fbf-a893-abd24b75dd95 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.852752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:01.889766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c3f32d-2a49-4b2a-8bce-016741c5c962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.890709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f32d-2a49-4b2a-8bce-016741c5c962 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.891012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f32d-2a49-4b2a-8bce-016741c5c962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.891366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c3f32d-2a49-4b2a-8bce-016741c5c962 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.892331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f32d-2a49-4b2a-8bce-016741c5c962 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.894321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c3f32d-2a49-4b2a-8bce-016741c5c962 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.908555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 915/1855] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:01 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:01.913236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:01.928914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2975792-71c0-427f-9dcd-396462eb220e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:01.935946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:01.937984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:01.938304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:01.939047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] PATH_INFO: `/v3/projects/4b45cba40d96425ab7e1cca987fdeb71/users/0311fbfbf7d64b1e9eebffa3f3b9600d/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:01.944007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b45cba40d96425ab7e1cca987fdeb71, user_id=0311fbfbf7d64b1e9eebffa3f3b9600d, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:01.949868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:01.971290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 941/1856] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:01 2022] PUT /identity/v3/projects/4b45cba40d96425ab7e1cca987fdeb71/users/0311fbfbf7d64b1e9eebffa3f3b9600d/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:03.003525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:03.004453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:03.005533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:03.032156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] MFA Rules not processed for user `0311fbfbf7d64b1e9eebffa3f3b9600d`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:03.033382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:03.166857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 916/1857] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:03 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:03.188703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:03.191418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:03.325027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:03.334240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e8ffa-2e9b-4a4a-89a0-17b6ba09bee8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:03.338768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00e7b7-7ede-4c46-9fe2-e7df2f48ed0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:03.339673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00e7b7-7ede-4c46-9fe2-e7df2f48ed0c service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:03.339957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00e7b7-7ede-4c46-9fe2-e7df2f48ed0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:03.340269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00e7b7-7ede-4c46-9fe2-e7df2f48ed0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:03.427383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00e7b7-7ede-4c46-9fe2-e7df2f48ed0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:03.428839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00e7b7-7ede-4c46-9fe2-e7df2f48ed0c service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:03.439640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 942/1858] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:04.287908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:04.290563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:04.409363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2419c-580b-44d5-90c8-0436e947e966 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:04.420221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:04.421419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:04.421928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:04.422425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] PATH_INFO: `/v3/users/5ebda67a890d4a64a0eb12ec67c7d3df` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:04.423868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ebda67a890d4a64a0eb12ec67c7d3df)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:04.427162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:04.514026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ebda67a890d4a64a0eb12ec67c7d3df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:04.525861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ebda67a890d4a64a0eb12ec67c7d3df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:04.527481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:04.544594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ebda67a890d4a64a0eb12ec67c7d3df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:04.553741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ebda67a890d4a64a0eb12ec67c7d3df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:04.554908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ebda67a890d4a64a0eb12ec67c7d3df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:04.571600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 917/1859] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:04 2022] DELETE /identity/v3/users/5ebda67a890d4a64a0eb12ec67c7d3df => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:04.573240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:04.573761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:04.573844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:04.611904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:04.613323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:04.748311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 943/1860] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:04.756932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:04.758486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:04.978282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b667-6ff9-4083-813c-185671b829d5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:04.982218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:04.982969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:04.983274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:04.983564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:05.004391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:05.006577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:05.022449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 918/1861] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:05.391235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:05.392746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:05.405431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4af58-66e2-4734-98e5-6ef2d365094f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:05.410495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:05.412075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:05.412455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:05.412966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] PATH_INFO: `/v3/projects/b1d8a1067b954225b585b6fc5b9b5843` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:05.423026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d8a1067b954225b585b6fc5b9b5843)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:05.425080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:05.507775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d8a1067b954225b585b6fc5b9b5843', 'request_id': 'req-c292b108-e23b-4278-8985-2741a1b20bdc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.514035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d8a1067b954225b585b6fc5b9b5843', 'request_id': 'req-c292b108-e23b-4278-8985-2741a1b20bdc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.521158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d8a1067b954225b585b6fc5b9b5843', 'request_id': 'req-c292b108-e23b-4278-8985-2741a1b20bdc'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.534118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] The token cache is being invalidate because project b1d8a1067b954225b585b6fc5b9b5843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:05.534410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.535028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.536745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 944/1862] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:05 2022] DELETE /identity/v3/projects/b1d8a1067b954225b585b6fc5b9b5843 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:05.541030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:05.550735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b63f683-caff-44d2-b8f5-7b981c454b50 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:05.608123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:05.609002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:05.609285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:05.609562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] PATH_INFO: `/v3/users/b36b0fdb1a3141418718d9875c84a0ae` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:05.610799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b36b0fdb1a3141418718d9875c84a0ae)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:05.612761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:05.669285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b36b0fdb1a3141418718d9875c84a0ae'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.677989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b36b0fdb1a3141418718d9875c84a0ae'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.690860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b36b0fdb1a3141418718d9875c84a0ae'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.696054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:05.700281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b36b0fdb1a3141418718d9875c84a0ae'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.701231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0d15a9e8-efc9-48f0-a1aa-aafcb0f6f69f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b36b0fdb1a3141418718d9875c84a0ae'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:05.707973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 919/1863] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:05 2022] DELETE /identity/v3/users/b36b0fdb1a3141418718d9875c84a0ae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:06.012098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.013575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.028901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c292b108-e23b-4278-8985-2741a1b20bdc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.070991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:06.111352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.112451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.112716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.113168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] PATH_INFO: `/v3/projects/0355c0bf331f49dc9c3e9c997725232c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.115197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0355c0bf331f49dc9c3e9c997725232c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.117411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.194472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0355c0bf331f49dc9c3e9c997725232c', 'request_id': 'req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.203505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0355c0bf331f49dc9c3e9c997725232c', 'request_id': 'req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.212841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0355c0bf331f49dc9c3e9c997725232c', 'request_id': 'req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.229073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] The token cache is being invalidate because project 0355c0bf331f49dc9c3e9c997725232c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:06.229437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.230235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:06.232987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 945/1864] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:06 2022] DELETE /identity/v3/projects/0355c0bf331f49dc9c3e9c997725232c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:06.239968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736716a7-82ec-41cf-906c-7eef13630a1f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.240263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736716a7-82ec-41cf-906c-7eef13630a1f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.240546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736716a7-82ec-41cf-906c-7eef13630a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.259448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-736716a7-82ec-41cf-906c-7eef13630a1f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:06.260663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-736716a7-82ec-41cf-906c-7eef13630a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.326829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.328322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.352429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 920/1865] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:06.360200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.360619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.361010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.384559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:06.385881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.399288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 921/1866] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:06.401920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.407833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baec5f9-b1f5-49f8-9a7d-b3bf3f17b975 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.411728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.412202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c2cb22-4df0-45ea-9054-f4f9fd664da7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.412496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.412780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.413007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.414783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.417961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.418085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.418387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.418735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.419552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:06.419941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.421757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.444603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 922/1867] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:06.445626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.457937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf578959-769c-436f-98f6-e82bd751fc2e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.462987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.463752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.464044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.464321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.465013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.466930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.498007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 923/1868] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:06 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:06.500106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.501567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.502122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.513463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 946/1869] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:06.572829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ea87c-5a44-4723-a461-7ce2227d5b45 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.579365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.580057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.580346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.580629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.581336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.583447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.612318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 924/1870] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:06 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:06.614939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.639153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274efdf1-c3f7-468d-a76e-bdf93a460c81 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.645815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.646493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.646780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.647061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.648072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.649760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.651283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:06.665972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 947/1871] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:06.669817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.696819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66fc0cb-c631-46cf-8962-959fbfc83c07 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.702799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cbe1c3-cad4-4c59-871e-aacf625a5eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.703726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe1c3-cad4-4c59-871e-aacf625a5eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.704192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe1c3-cad4-4c59-871e-aacf625a5eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.704654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cbe1c3-cad4-4c59-871e-aacf625a5eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.705649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbe1c3-cad4-4c59-871e-aacf625a5eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.707310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cbe1c3-cad4-4c59-871e-aacf625a5eb8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.722708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 925/1872] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:06.723997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.735033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a357e68-113c-4db6-b023-707268da29cf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.740784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.741770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.742195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.742477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] PATH_INFO: `/v3/projects/58680c3a82c74c43ac42b62014f98e57/users/ce259ef5044548899c114da18e673060/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.760175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58680c3a82c74c43ac42b62014f98e57, user_id=ce259ef5044548899c114da18e673060, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.763188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:06.779683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 948/1873] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:06 2022] PUT /identity/v3/projects/58680c3a82c74c43ac42b62014f98e57/users/ce259ef5044548899c114da18e673060/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:06.791001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.791413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.791811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.819710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] MFA Rules not processed for user `ce259ef5044548899c114da18e673060`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:06.821359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:06.843075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:06.899267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 926/1874] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:06.915585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:06.977344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194ea15-9761-47b6-ab0d-b7407927c845 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:06.981727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d3916-cbb0-4d8d-b32e-b3563baee511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:06.982684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3916-cbb0-4d8d-b32e-b3563baee511 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:06.983073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3916-cbb0-4d8d-b32e-b3563baee511 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:06.983482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d3916-cbb0-4d8d-b32e-b3563baee511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:06.992900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d3916-cbb0-4d8d-b32e-b3563baee511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:06.994910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d3916-cbb0-4d8d-b32e-b3563baee511 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.005723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 949/1875] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:07.019143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.021603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.098559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b9a99-d2a2-4fe6-bb30-fcb0ef4c1e96 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:07.103289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:07.104164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.104442 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.104712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] PATH_INFO: `/v3/users/813050e148ca4e6b8eb7e52a363736df` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.105920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813050e148ca4e6b8eb7e52a363736df)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:07.118388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.189954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813050e148ca4e6b8eb7e52a363736df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.197918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813050e148ca4e6b8eb7e52a363736df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.211260 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813050e148ca4e6b8eb7e52a363736df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.218196 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813050e148ca4e6b8eb7e52a363736df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.219031 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813050e148ca4e6b8eb7e52a363736df'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.237768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 927/1876] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:07 2022] DELETE /identity/v3/users/813050e148ca4e6b8eb7e52a363736df => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:07.240623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.241367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.241667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.260809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:07.262073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.359156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 950/1877] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:07.369343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.370955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.419564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:07.437016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872be7aa-0a2b-4f3c-9d5e-64a073d88860 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:07.440762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:07.441406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.441689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.442016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.450902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:07.452396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.461736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 928/1878] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:07.824856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.826627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:07.841509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c03fde-fcd3-4a10-b6c8-e07d0580b9f2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:07.847237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:07.849438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:07.849929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:07.850285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] PATH_INFO: `/v3/projects/4d3f6de82dc74d38a6ca406bd320f448` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:07.852118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d3f6de82dc74d38a6ca406bd320f448)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:07.854484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:07.907925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d3f6de82dc74d38a6ca406bd320f448', 'request_id': 'req-8d7b6f07-d095-412a-b309-473dfab2a042'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.914618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d3f6de82dc74d38a6ca406bd320f448', 'request_id': 'req-8d7b6f07-d095-412a-b309-473dfab2a042'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.921246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d3f6de82dc74d38a6ca406bd320f448', 'request_id': 'req-8d7b6f07-d095-412a-b309-473dfab2a042'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.929559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:07.935075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] The token cache is being invalidate because project 4d3f6de82dc74d38a6ca406bd320f448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:07.935431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.936702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:07.939680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 951/1879] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:07 2022] DELETE /identity/v3/projects/4d3f6de82dc74d38a6ca406bd320f448 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:07.944757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:07.954249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10974d-fdff-4d37-92b4-0a1b13b61e25 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.010459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.011276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.011560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.011843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] PATH_INFO: `/v3/users/c495c2611f29499e9b3f967b8c09ee55` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.013035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c495c2611f29499e9b3f967b8c09ee55)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.014960 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.060036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c495c2611f29499e9b3f967b8c09ee55'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.068407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c495c2611f29499e9b3f967b8c09ee55'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.080449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c495c2611f29499e9b3f967b8c09ee55'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.088037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c495c2611f29499e9b3f967b8c09ee55'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.088691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c495c2611f29499e9b3f967b8c09ee55'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.095481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 929/1880] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:07 2022] DELETE /identity/v3/users/c495c2611f29499e9b3f967b8c09ee55 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:08.390725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:08.393705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.423485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:08.425831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.528130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7b6f07-d095-412a-b309-473dfab2a042 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.601692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5dcca69-0c6f-4c8d-b937-5f0cc453a0b9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.616518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.617356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.617635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.618010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] PATH_INFO: `/v3/projects/65d16706dfba48c3a30362f9027a6df5` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.619342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] RBAC: Authorizing `identity:delete_project(project_id=65d16706dfba48c3a30362f9027a6df5)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.621403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.624836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:08.636270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.638040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.638449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.639076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.646664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.658146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.660012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.670980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 952/1881] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:08.672032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:08.673842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.686310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d0de9-c5b7-4d5d-b2c6-5b4a4ee702f0 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.690990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65d16706dfba48c3a30362f9027a6df5', 'request_id': 'req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.709176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65d16706dfba48c3a30362f9027a6df5', 'request_id': 'req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.719281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65d16706dfba48c3a30362f9027a6df5', 'request_id': 'req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.738815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] The token cache is being invalidate because project 65d16706dfba48c3a30362f9027a6df5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:08.739780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.741123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.745640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 930/1882] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:08 2022] DELETE /identity/v3/projects/65d16706dfba48c3a30362f9027a6df5 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:08.746891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:08.748380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.764070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.764758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.765042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.765396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.767390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:08.778248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.779889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04be7ba-3ec6-41cc-b2fe-c13029f009f7 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.789695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 953/1883] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:08.840580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52dc85-49b7-4a25-aff9-9a0f3aefc72c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:08.846302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:08.847397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:08.847779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:08.848152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] PATH_INFO: `/v3/users/643b3eb172ed4f40bb856d4f37b24c3b` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:08.850816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=643b3eb172ed4f40bb856d4f37b24c3b)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:08.853171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:08.946485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '643b3eb172ed4f40bb856d4f37b24c3b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.956555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '643b3eb172ed4f40bb856d4f37b24c3b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:08.988500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '643b3eb172ed4f40bb856d4f37b24c3b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.002859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '643b3eb172ed4f40bb856d4f37b24c3b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.003752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '643b3eb172ed4f40bb856d4f37b24c3b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:09.011555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 931/1884] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:08 2022] DELETE /identity/v3/users/643b3eb172ed4f40bb856d4f37b24c3b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:10.376832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9374196-e4ca-46bb-a7e4-9b454a736d78 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.377296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9374196-e4ca-46bb-a7e4-9b454a736d78 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.377512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9374196-e4ca-46bb-a7e4-9b454a736d78 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.379570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 954/1885] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:42:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:42:10.451358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.452886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.481575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:10.589629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86581d8-d64c-4ec7-8134-eb08b396fdf5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.596085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.596973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.597256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.597545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] PATH_INFO: `/v3/projects/0141955f05564a55acea7197ed67dd89` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.599417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0141955f05564a55acea7197ed67dd89)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:10.601936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:10.674382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0141955f05564a55acea7197ed67dd89', 'request_id': 'req-885f200d-569b-4093-a835-136dde75ebca'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.688511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0141955f05564a55acea7197ed67dd89', 'request_id': 'req-885f200d-569b-4093-a835-136dde75ebca'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.690037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c17ec-27ca-48d1-9c94-58f8dad6f1bf None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.690326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c17ec-27ca-48d1-9c94-58f8dad6f1bf None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.690605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188c17ec-27ca-48d1-9c94-58f8dad6f1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.705163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0141955f05564a55acea7197ed67dd89', 'request_id': 'req-885f200d-569b-4093-a835-136dde75ebca'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.728274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-188c17ec-27ca-48d1-9c94-58f8dad6f1bf None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:10.729513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-188c17ec-27ca-48d1-9c94-58f8dad6f1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.735438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] The token cache is being invalidate because project 0141955f05564a55acea7197ed67dd89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:10.735729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.736661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-885f200d-569b-4093-a835-136dde75ebca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:10.738296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 932/1886] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:10 2022] DELETE /identity/v3/projects/0141955f05564a55acea7197ed67dd89 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:10.744952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.745243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.745589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.792135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:10.793143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.832041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:10.856924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 955/1887] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:10.888791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 933/1888] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:10.890668 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.891169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.891552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.916473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:10.918209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.954157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 956/1889] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:10.966984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.978436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:10.980561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6644d51-5852-4a22-b9c0-a33b2f9ed6f5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:10.981990 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:10.990562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:10.990562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:10.990562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:10.990562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:10.991136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:10.995461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.001431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.008914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f1a5c7-bf00-4c9d-8638-7da13af2ceae None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.015761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.016933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.017406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.017830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.018132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 934/1890] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:11.023063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.031517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.033244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.045413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916191ce-9593-4afa-b274-42ce03aa9827 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.046796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 957/1891] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:11.069777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.070672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.071261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.071862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.072779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.075476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.136952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 935/1892] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/projects => generated 361 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:11.137549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.153079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c01a30e-ac69-4d5a-a60d-b6e554d34415 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.276124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.276805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.277090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.277367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.278071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.280141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.306565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 958/1893] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/users => generated 281 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:11.311749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.328028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574c04d-a022-4f17-81bb-e29faf91314e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.335457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.336071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.336547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.336916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.337703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.339698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.341327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:11.355990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 936/1894] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:11.356461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.368544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b500a3bd-989e-4839-b3fc-e28d471ad490 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.375328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8988e8-c094-4171-afa7-6943987d5def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.376092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8988e8-c094-4171-afa7-6943987d5def admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.376373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8988e8-c094-4171-afa7-6943987d5def admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.376883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8988e8-c094-4171-afa7-6943987d5def admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.378290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8988e8-c094-4171-afa7-6943987d5def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.380879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8988e8-c094-4171-afa7-6943987d5def admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.395758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 959/1895] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:11.397310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.408140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaa8f92-7d66-4dd8-9b37-08b70f14a292 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.418276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a80eaf-e1fe-46fd-b3e3-77738a0edf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.419640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a80eaf-e1fe-46fd-b3e3-77738a0edf8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.419924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a80eaf-e1fe-46fd-b3e3-77738a0edf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.420255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a80eaf-e1fe-46fd-b3e3-77738a0edf8f admin admin] PATH_INFO: `/v3/projects/b0a868ef981c4eb28c1e30b05c997b25/users/1414a596005b461ca48254ab5d84c61a/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.445560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a80eaf-e1fe-46fd-b3e3-77738a0edf8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0a868ef981c4eb28c1e30b05c997b25, user_id=1414a596005b461ca48254ab5d84c61a, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.453156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a80eaf-e1fe-46fd-b3e3-77738a0edf8f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.475252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 937/1896] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:11 2022] PUT /identity/v3/projects/b0a868ef981c4eb28c1e30b05c997b25/users/1414a596005b461ca48254ab5d84c61a/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:11.482982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.483340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.483629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.525114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] MFA Rules not processed for user `1414a596005b461ca48254ab5d84c61a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:11.526513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.686863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34085-1b60-42f0-b96f-6cb03a7a84ef None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.687342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34085-1b60-42f0-b96f-6cb03a7a84ef None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.687898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac34085-1b60-42f0-b96f-6cb03a7a84ef None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.689627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 938/1897] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:42:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:42:11.690141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 960/1898] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:11.700964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.701246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.701595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.736394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:11.737678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.742531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.745323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.850984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:11.857986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 939/1899] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:42:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:11.877001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:11.878427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:11.892605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592acdb-d436-4809-93bd-d088e5a65ac1 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.898486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc33b25-4280-4782-b993-b2628474d9c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.899305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8725cd-dadd-4919-a8b8-a7023fc1576a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:11.899611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc33b25-4280-4782-b993-b2628474d9c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.900342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc33b25-4280-4782-b993-b2628474d9c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.900994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc33b25-4280-4782-b993-b2628474d9c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.913185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:11.914236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:11.914769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:11.915197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:11.932796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.934666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.946274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc33b25-4280-4782-b993-b2628474d9c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:11.947810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc33b25-4280-4782-b993-b2628474d9c3 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:11.950918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 940/1900] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:11.968411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 961/1901] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:12.002990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:12.098779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1421d6d-767f-4196-a605-b1e0187b82c1 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:12.103761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:12.104660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.105125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.105429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] PATH_INFO: `/v3/users/1414a596005b461ca48254ab5d84c61a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.110389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1414a596005b461ca48254ab5d84c61a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:12.112493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:12.171878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1414a596005b461ca48254ab5d84c61a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.176952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:12.187573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1414a596005b461ca48254ab5d84c61a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.204252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1414a596005b461ca48254ab5d84c61a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.213781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1414a596005b461ca48254ab5d84c61a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.214724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1414a596005b461ca48254ab5d84c61a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.221627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 941/1902] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:12 2022] DELETE /identity/v3/users/1414a596005b461ca48254ab5d84c61a => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:12.226811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.227099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.227415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.255809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:12.257151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:12.323414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 962/1903] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:12.334944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:12.336570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:12.398890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6031ac18-165a-4b3a-9686-67448a1646b5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:12.404038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7599300-73d7-4c78-839e-b7401d78ec6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:12.405172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7599300-73d7-4c78-839e-b7401d78ec6f service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.405458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7599300-73d7-4c78-839e-b7401d78ec6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.405737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7599300-73d7-4c78-839e-b7401d78ec6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.422526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7599300-73d7-4c78-839e-b7401d78ec6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:12.422526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7599300-73d7-4c78-839e-b7401d78ec6f service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:12.428458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 942/1904] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:12.790982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:12.792685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:12.805054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa949b-4d4f-49ad-b734-ef0b8dd9bf81 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:12.809891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:12.810769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.811053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.811360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] PATH_INFO: `/v3/projects/b0a868ef981c4eb28c1e30b05c997b25` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.812548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0a868ef981c4eb28c1e30b05c997b25)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:12.814470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:12.861452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0a868ef981c4eb28c1e30b05c997b25', 'request_id': 'req-24d698a2-8ee1-4d81-a638-247f10fc9127'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.867466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0a868ef981c4eb28c1e30b05c997b25', 'request_id': 'req-24d698a2-8ee1-4d81-a638-247f10fc9127'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.873541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0a868ef981c4eb28c1e30b05c997b25', 'request_id': 'req-24d698a2-8ee1-4d81-a638-247f10fc9127'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.886005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] The token cache is being invalidate because project b0a868ef981c4eb28c1e30b05c997b25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:12.886290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.886912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-24d698a2-8ee1-4d81-a638-247f10fc9127 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:12.888446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 963/1905] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:12 2022] DELETE /identity/v3/projects/b0a868ef981c4eb28c1e30b05c997b25 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:12.893333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.893815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.893906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:12.910681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:12.911823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:12.972534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 943/1906] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:12.980696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:12.981134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:12.981632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.005868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:13.007100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:13.015059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 964/1907] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:13.019148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.028596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcbb63-8026-4b25-b9c6-1ee9312189d1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.033133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.033786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.034072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.034417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.035063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:13.035528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.037127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.048749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 944/1908] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:13.059946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.062724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433b7f77-cc53-403b-b9ef-18c4e4af450b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.067963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.068617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.068946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.069227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.069930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.071861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.098683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 965/1909] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:13 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:13.102654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.112608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af395b2-9959-477b-ac9a-b7072c23083e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.167867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.168461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.168775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.169055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.169774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.171719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.179192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:13.196217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 945/1910] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:13 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:13.200272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.210421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87223d7b-96af-49bf-b647-eb3587465ad6 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.214910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.215597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.215880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.216155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.216876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.218450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.229569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 966/1911] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:13.233428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.243780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d79913-64fc-42f8-b382-58ec5ca182eb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.248510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85051b6d-2dfd-43de-b05c-cd4c3e6b5b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.249150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85051b6d-2dfd-43de-b05c-cd4c3e6b5b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.249434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85051b6d-2dfd-43de-b05c-cd4c3e6b5b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.249711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85051b6d-2dfd-43de-b05c-cd4c3e6b5b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.250434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85051b6d-2dfd-43de-b05c-cd4c3e6b5b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.252061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85051b6d-2dfd-43de-b05c-cd4c3e6b5b51 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.261780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 946/1912] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:13.266029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.275887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3789a4-df4b-41fd-9716-f89d068f9b63 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.280448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.281349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.281628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.281903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] PATH_INFO: `/v3/projects/5e6d51dece9847669e9a666633ce634d/users/46097d36b6fc4df5b510d279a4648eb7/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.297199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e6d51dece9847669e9a666633ce634d, user_id=46097d36b6fc4df5b510d279a4648eb7, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.299921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.312925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 967/1913] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:13 2022] PUT /identity/v3/projects/5e6d51dece9847669e9a666633ce634d/users/46097d36b6fc4df5b510d279a4648eb7/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:13.317872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44c88a-11ef-4414-9710-aca010b987be None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.318163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44c88a-11ef-4414-9710-aca010b987be None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.318467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd44c88a-11ef-4414-9710-aca010b987be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.335580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-dd44c88a-11ef-4414-9710-aca010b987be None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:13.336763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dd44c88a-11ef-4414-9710-aca010b987be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:13.394085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 947/1914] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:13.403529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:13.405186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:13.410457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:13.470444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff04398-e1ed-4eaa-bf73-29a8f638992a admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:13.474283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:13.474974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:13.475305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:13.475602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:13.485649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:13.487285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:13.497648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 968/1915] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:15.101953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb0bf2-60ce-4638-9870-c4677f3b362a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.104177 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb0bf2-60ce-4638-9870-c4677f3b362a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.104736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fb0bf2-60ce-4638-9870-c4677f3b362a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.188127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-22fb0bf2-60ce-4638-9870-c4677f3b362a None None] MFA Rules not processed for user `46097d36b6fc4df5b510d279a4648eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:15.191828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-22fb0bf2-60ce-4638-9870-c4677f3b362a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:15.330929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 948/1916] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:15.356072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:15.358782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:15.460944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef2995a-58df-413a-bdae-4b0e74fcf710 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:15.465690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:15.466379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:15.466806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:15.467091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:15.477971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:15.479783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:15.491957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 969/1917] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:15 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:18.010721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.011343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.011343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.032928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] MFA Rules not processed for user `46097d36b6fc4df5b510d279a4648eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:18.034121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.042428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 949/1918] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:18.206042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:18.207515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.233224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb009a-e81b-4af2-84c0-8e9821fe4953 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:18.239070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:18.239976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.240261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.240535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] PATH_INFO: `/v3/users/2a58579bdb2f453591bac996511af7f5` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.241812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a58579bdb2f453591bac996511af7f5)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:18.244035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:18.328645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a58579bdb2f453591bac996511af7f5'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.338910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a58579bdb2f453591bac996511af7f5'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.356459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a58579bdb2f453591bac996511af7f5'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.357511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a58579bdb2f453591bac996511af7f5'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.358677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bf8a2e2a-e80d-4f0c-9018-a1df705a97d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a58579bdb2f453591bac996511af7f5'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.392147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:18.393774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.402501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 970/1919] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:18 2022] DELETE /identity/v3/users/2a58579bdb2f453591bac996511af7f5 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:18.404315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.404768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.405197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.412923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4967c43a-0c8f-42fc-8604-c387755a1532 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:18.425352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:18.427032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.448502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:18.501561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:18.502911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.503484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.503861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] PATH_INFO: `/v3/users/ce259ef5044548899c114da18e673060` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.505675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce259ef5044548899c114da18e673060)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:18.507788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:18.514326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 971/1920] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:18.522858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:18.524864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:18.527305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.602516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2526f-a477-4248-852a-15d7d19f629f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:18.606032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce259ef5044548899c114da18e673060'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.607316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445362ca-59e9-4ed2-b24a-3e84bfa94ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:18.608092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445362ca-59e9-4ed2-b24a-3e84bfa94ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.608676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445362ca-59e9-4ed2-b24a-3e84bfa94ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.609060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445362ca-59e9-4ed2-b24a-3e84bfa94ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.618186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce259ef5044548899c114da18e673060'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.623662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445362ca-59e9-4ed2-b24a-3e84bfa94ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:18.625700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445362ca-59e9-4ed2-b24a-3e84bfa94ba6 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:18.641988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 972/1921] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:18.644903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce259ef5044548899c114da18e673060'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.653467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce259ef5044548899c114da18e673060'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.654213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce259ef5044548899c114da18e673060'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:18.663406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 950/1922] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:18 2022] DELETE /identity/v3/users/ce259ef5044548899c114da18e673060 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:18.668070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:18.669066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:18.669422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:18.707081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:18.721969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.859752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 973/1923] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:18.866346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:18.868248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:18.996330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f74ff-7699-4eae-9e93-42f76b9a5708 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.000431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea78648-c482-437a-986b-d5d42decefad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.001130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea78648-c482-437a-986b-d5d42decefad service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.001554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea78648-c482-437a-986b-d5d42decefad service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.001836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea78648-c482-437a-986b-d5d42decefad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.016343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea78648-c482-437a-986b-d5d42decefad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.017813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea78648-c482-437a-986b-d5d42decefad service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.033600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 951/1924] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:19.066632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.068127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.080469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58067ffe-4e35-4fb8-9077-506ca4da10ea None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.086788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.088556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.088992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.089349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] PATH_INFO: `/v3/projects/36fd432b4335442990c31d60931f1b8b` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.090849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36fd432b4335442990c31d60931f1b8b)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.093487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.179885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36fd432b4335442990c31d60931f1b8b', 'request_id': 'req-b391945d-9370-4b5c-b17f-605786dc267c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.188021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36fd432b4335442990c31d60931f1b8b', 'request_id': 'req-b391945d-9370-4b5c-b17f-605786dc267c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.190042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:19.198182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36fd432b4335442990c31d60931f1b8b', 'request_id': 'req-b391945d-9370-4b5c-b17f-605786dc267c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.213268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] The token cache is being invalidate because project 36fd432b4335442990c31d60931f1b8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:19.213921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.214271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b391945d-9370-4b5c-b17f-605786dc267c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.216447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 974/1925] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:19 2022] DELETE /identity/v3/projects/36fd432b4335442990c31d60931f1b8b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:19.221897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.221897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.221897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.245754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:19.247504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.315287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 952/1926] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:19.332906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.334665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.334958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.360137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:19.361349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.376196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 975/1927] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:19.380907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.395824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18262ff-8656-4175-b961-c0f14b40bb2f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.401476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.402887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.402887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.403087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.403765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:19.404144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.405932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.427096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 953/1928] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:19.428006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.437699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb4ce2f-f3ce-4e1c-a611-33679d32d1d2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.444478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.445208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.445851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.445968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.448026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.449957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.483291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 976/1929] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:19.487013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.499538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd473ea2-a69a-4008-85af-1ee2d0649548 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.508068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:19.558829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.560633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.601945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.604121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.604121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.604121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.604121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.605784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.631351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 954/1930] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/users => generated 283 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:19.635729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.646730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0f0e8a-dfb4-402f-bbbc-3a0f079101e4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.653646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.654403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.654761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.655107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.655945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.657751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.664631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c492a24c-9309-49f2-946b-5b25b001802e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.670111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.670367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 955/1931] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:19.671484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.671775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.672050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] PATH_INFO: `/v3/projects/58680c3a82c74c43ac42b62014f98e57` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.674144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58680c3a82c74c43ac42b62014f98e57)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.674526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.676088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.685052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7ca56-1d1c-4516-a722-87d5005f7778 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.690174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.690868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.691159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.691466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.692889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.694490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.705656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 956/1932] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:19.710962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:19.722675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fcd0aa-319e-41ed-a534-61f184cafdaa admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:19.728994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58680c3a82c74c43ac42b62014f98e57', 'request_id': 'req-7b446e27-783a-4243-9cf6-ff0256980375'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.735678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58680c3a82c74c43ac42b62014f98e57', 'request_id': 'req-7b446e27-783a-4243-9cf6-ff0256980375'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.744059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58680c3a82c74c43ac42b62014f98e57', 'request_id': 'req-7b446e27-783a-4243-9cf6-ff0256980375'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.759447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] The token cache is being invalidate because project 58680c3a82c74c43ac42b62014f98e57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:19.759747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.760422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7b446e27-783a-4243-9cf6-ff0256980375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:19.762104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 977/1933] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:19 2022] DELETE /identity/v3/projects/58680c3a82c74c43ac42b62014f98e57 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:19.766732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ec0d-a84a-43b6-9338-70f104b308c8 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.767055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ec0d-a84a-43b6-9338-70f104b308c8 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.767341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46ec0d-a84a-43b6-9338-70f104b308c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.797315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-bb46ec0d-a84a-43b6-9338-70f104b308c8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:19.798310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bb46ec0d-a84a-43b6-9338-70f104b308c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.812715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846da08a-c1cc-414e-9063-dc1537f99a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:19.814303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da08a-c1cc-414e-9063-dc1537f99a93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.814600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da08a-c1cc-414e-9063-dc1537f99a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.815689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846da08a-c1cc-414e-9063-dc1537f99a93 admin admin] PATH_INFO: `/v3/projects/3b71e809a3b8407eb499c44f95264fdb/users/fbda96afb3a04916906e63f82c15da1c/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.840691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846da08a-c1cc-414e-9063-dc1537f99a93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b71e809a3b8407eb499c44f95264fdb, user_id=fbda96afb3a04916906e63f82c15da1c, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:19.845067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846da08a-c1cc-414e-9063-dc1537f99a93 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:19.869740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 957/1934] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:19 2022] PUT /identity/v3/projects/3b71e809a3b8407eb499c44f95264fdb/users/fbda96afb3a04916906e63f82c15da1c/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:19.886166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.886449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.886745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.887464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:19.913836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 978/1935] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:19.921597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:19.922141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:19.922678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:19.959004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] MFA Rules not processed for user `fbda96afb3a04916906e63f82c15da1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:19.959191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:19.960544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:19.960658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:20.067761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:20.077301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 979/1936] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:20.082577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.088063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 958/1937] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:19 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:20.093310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.096051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:20.102706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3c093-8ad1-4bd5-a003-c5086faa8e8a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.108150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.109339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.109470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.109849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.110046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3545e-af83-4eb0-9ff5-d6aab2e6c87a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.111066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:20.111602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.113373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.115172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.115875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.116172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.116449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.117146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.119248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.151199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 980/1938] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:20.156168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 959/1939] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/projects => generated 365 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:20.156431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.157300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.170653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90112b85-8fbc-4bae-9e47-e27464ec26c2 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.257678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69bbbc-8f4b-457e-8fa5-75098f80549f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.264069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.264579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.265007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.265352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.265634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.266172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.266355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.266830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.267338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.268389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.268510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.270823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.300021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 960/1940] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:20.301028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.305296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 981/1941] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:20.310087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.319414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083c9713-f6d6-4aa0-b71a-fd5ee7f4f887 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.427507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.428809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.429920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.430318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.431176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.432920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.452952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b658bd-2841-46f9-9c78-86b9fbdaf48c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.455288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 961/1942] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:20.457267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.469972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.470957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.471443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.471863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.472711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.475939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77e9e-2a1b-4bce-8d79-b9da067e976a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.478785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.482480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.483443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.483701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.483977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] PATH_INFO: `/v3/projects/f083c3e502924f56b479c4e8421c26ea/users/d99b7cc5aaf24d259cebe9c4fce361a0/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.491347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:20.509711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f083c3e502924f56b479c4e8421c26ea, user_id=d99b7cc5aaf24d259cebe9c4fce361a0, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.513252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.514034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 982/1943] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:20.518229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.535114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 962/1944] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:20 2022] PUT /identity/v3/projects/f083c3e502924f56b479c4e8421c26ea/users/d99b7cc5aaf24d259cebe9c4fce361a0/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:20.538913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28466ea6-43e1-4a4b-bb84-4166fad1a754 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.539880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.617409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:20.717480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.718232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.718588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.718952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.721766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.722844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.727903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8bbfd3-74e6-49c5-bec5-d10cb7906494 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.739541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 983/1945] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:20.741198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.747165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.747898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.748179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.748455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.757878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.766975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee86bd47-af3c-4b48-a520-943698db8971 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.767982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.778702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.781790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.782233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.782391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.786629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.791551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.795855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 963/1946] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:20.800668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.825143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 984/1947] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:20.826504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:20.835101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cef084-16c6-4f0c-8453-a40c6348b98b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.841299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3233d8b8-8b59-4592-bdea-b1d9de4eec5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.843754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233d8b8-8b59-4592-bdea-b1d9de4eec5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.844396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233d8b8-8b59-4592-bdea-b1d9de4eec5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.844894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3233d8b8-8b59-4592-bdea-b1d9de4eec5d admin admin] PATH_INFO: `/v3/projects/f083c3e502924f56b479c4e8421c26ea/users/d99b7cc5aaf24d259cebe9c4fce361a0/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.849086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233d8b8-8b59-4592-bdea-b1d9de4eec5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f083c3e502924f56b479c4e8421c26ea, user_id=d99b7cc5aaf24d259cebe9c4fce361a0, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.850385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27b56c-1b91-408d-9a30-5e74a71035a2 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:20.852426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3233d8b8-8b59-4592-bdea-b1d9de4eec5d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.863346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0bac28-0d7f-459c-a359-5be8e90b0851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:20.865726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bac28-0d7f-459c-a359-5be8e90b0851 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.866017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bac28-0d7f-459c-a359-5be8e90b0851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.866303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0bac28-0d7f-459c-a359-5be8e90b0851 admin admin] PATH_INFO: `/v3/projects/dfdad00feabb44c6b95fa0d5b328fb5f/users/aff37b3f7c864aa0b7f95626c57f185d/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.867825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 964/1948] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:20 2022] PUT /identity/v3/projects/f083c3e502924f56b479c4e8421c26ea/users/d99b7cc5aaf24d259cebe9c4fce361a0/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:20.878302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.879475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.880856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.910519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bac28-0d7f-459c-a359-5be8e90b0851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdad00feabb44c6b95fa0d5b328fb5f, user_id=aff37b3f7c864aa0b7f95626c57f185d, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:20.913298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0bac28-0d7f-459c-a359-5be8e90b0851 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:20.943332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 985/1949] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:20 2022] PUT /identity/v3/projects/dfdad00feabb44c6b95fa0d5b328fb5f/users/aff37b3f7c864aa0b7f95626c57f185d/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:20.955703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:20.955996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:20.956277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:20.972375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] MFA Rules not processed for user `d99b7cc5aaf24d259cebe9c4fce361a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:20.993906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:21.014123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] MFA Rules not processed for user `aff37b3f7c864aa0b7f95626c57f185d`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:21.017021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:21.188389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 986/1950] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:21.190547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.209538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 965/1951] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:20 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:21.246769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.248682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:21.276511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:21.302958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b03361-f4ec-4902-a00d-99ed30f0619f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.308284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.309234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.309610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.309982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.311089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.313367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.347892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 987/1952] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:21 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:21.352782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.375412 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b6bfed-7d9b-4785-958e-f0d2340d4864 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.456082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.457189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.457482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.457947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.469705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:21.472245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fbf63f-5262-454b-b1e2-699af8f8fb82 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.479484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.480196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.480449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.480755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.481423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.483993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.508745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 988/1953] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:21 2022] POST /identity/v3/users => generated 289 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:21.512590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.523313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f71f0-b608-4107-aaa3-e5a8e985427d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.528370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.529150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.529530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.529868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.530672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.532360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.541381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.542833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.544395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 989/1954] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:21.547798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.553817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 966/1955] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:21 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:21.558196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f28454-ed1f-4037-9ff6-e95c06e1444b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.562958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a70a8d-c0fe-485e-b045-8440ffee5a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.563729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a70a8d-c0fe-485e-b045-8440ffee5a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.564014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a70a8d-c0fe-485e-b045-8440ffee5a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.564286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a70a8d-c0fe-485e-b045-8440ffee5a37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.565002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a70a8d-c0fe-485e-b045-8440ffee5a37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.566558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a70a8d-c0fe-485e-b045-8440ffee5a37 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.578722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 990/1956] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:21.583119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.594241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd41fc98-9283-41b1-9ee4-af6460659237 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.600785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.601744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.602099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.602445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] PATH_INFO: `/v3/projects/dbef7dad1b564f0eabd5f711608429bb/users/97e6608cd59e42d390bfd95eb91396cb/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.618618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbef7dad1b564f0eabd5f711608429bb, user_id=97e6608cd59e42d390bfd95eb91396cb, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.621425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.635686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 967/1957] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:21 2022] PUT /identity/v3/projects/dbef7dad1b564f0eabd5f711608429bb/users/97e6608cd59e42d390bfd95eb91396cb/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:21.648185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.648515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.648791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.687349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] MFA Rules not processed for user `97e6608cd59e42d390bfd95eb91396cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:21.688552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:21.787265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 991/1958] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:21.789320 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:21.905305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41594964-6ac9-4cfe-be2f-2a3d4ec63b4f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:21.918779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:21.919546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:21.919836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:21.920118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:21.921175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:21.923155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:21.981414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 968/1959] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:21 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:21.985406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.003272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfa60ce-4337-45a4-9c00-2bf201a954d5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.023612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:22.070203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.070900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.071187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.071518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.072233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.074184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.098495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 992/1960] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:21 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:22.105021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.106520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.117929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08ce19f-4851-4614-877e-c3350b3f449b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.119770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d5bfa5-1df2-40cf-8cd4-d1abffd36441 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.122945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.123672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.123955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.124231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.124766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.124969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.125652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.125936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.126232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] PATH_INFO: `/v3/users/fbda96afb3a04916906e63f82c15da1c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.126608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.127962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbda96afb3a04916906e63f82c15da1c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.130057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.138699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 969/1961] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:22.142706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.154668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a495bdb-8c29-484b-b064-23aea798b98e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.160119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.161614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.161887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.162987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] PATH_INFO: `/v3/projects/d18f3acf3e9c4e46964b14bede4ad06a/users/502dec5d05aa477e8c84b7a3f00005f6/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.192742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbda96afb3a04916906e63f82c15da1c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:22.192885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18f3acf3e9c4e46964b14bede4ad06a, user_id=502dec5d05aa477e8c84b7a3f00005f6, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.196863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.201984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbda96afb3a04916906e63f82c15da1c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:22.210301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbda96afb3a04916906e63f82c15da1c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:22.211147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbda96afb3a04916906e63f82c15da1c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:22.214556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 970/1962] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:22 2022] PUT /identity/v3/projects/d18f3acf3e9c4e46964b14bede4ad06a/users/502dec5d05aa477e8c84b7a3f00005f6/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:22.215527 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e6f8127a-c5b2-42c5-88ca-260b2f2ff886 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbda96afb3a04916906e63f82c15da1c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:22.215662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.217315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.236512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 993/1963] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:22 2022] DELETE /identity/v3/users/fbda96afb3a04916906e63f82c15da1c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:22.241684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598dfe3-0a94-42be-9726-d2a12f7df159 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.242183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598dfe3-0a94-42be-9726-d2a12f7df159 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.242671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598dfe3-0a94-42be-9726-d2a12f7df159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.255675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:22.262476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1598dfe3-0a94-42be-9726-d2a12f7df159 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:22.263526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1598dfe3-0a94-42be-9726-d2a12f7df159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.290068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dc34d-519a-4b61-ab45-2e1b456282c8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.337970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 994/1964] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:22.341516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a041aa-61bb-44a5-8c34-ec7fd3062951 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.341516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a041aa-61bb-44a5-8c34-ec7fd3062951 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.341516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a041aa-61bb-44a5-8c34-ec7fd3062951 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.342889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 995/1965] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:42:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:42:22.352678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.353044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.353653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.359791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.360542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.360892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.361347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.362198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.363989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.378463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 971/1966] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:22 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:22.378463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.379972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:22.380373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.381188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.411609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:22.450368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffac80-ecea-4755-83ad-88b1493b99e4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.456321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.457071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.457427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.457811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.470231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.471748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.481889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 996/1967] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:42:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:22.483464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.503614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96027c60-eecd-4d04-86cd-049b2dd39618 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.507728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 972/1968] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:22.530228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2198da4e-ea07-4467-bab0-7855affca923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.531147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198da4e-ea07-4467-bab0-7855affca923 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.531488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198da4e-ea07-4467-bab0-7855affca923 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.531773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2198da4e-ea07-4467-bab0-7855affca923 admin admin] PATH_INFO: `/v3/projects/d18f3acf3e9c4e46964b14bede4ad06a/users/502dec5d05aa477e8c84b7a3f00005f6/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.536966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198da4e-ea07-4467-bab0-7855affca923 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d18f3acf3e9c4e46964b14bede4ad06a, user_id=502dec5d05aa477e8c84b7a3f00005f6, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.539505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2198da4e-ea07-4467-bab0-7855affca923 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.545493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.547407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.553815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 997/1969] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:22 2022] PUT /identity/v3/projects/d18f3acf3e9c4e46964b14bede4ad06a/users/502dec5d05aa477e8c84b7a3f00005f6/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:22.563726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.564332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.564820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.586741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1063f5a9-e74e-419f-a081-91b455a26bfe service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.612950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] MFA Rules not processed for user `502dec5d05aa477e8c84b7a3f00005f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:22.614912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.712511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.713449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.713736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.714012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.741898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:22.743563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:22.783348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 998/1970] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:22 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:22.791607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 973/1971] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:22.811003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:22.813078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.912095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79faa3c0-3c5b-49ac-ae97-918a46fd17fd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:22.916917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:22.917064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:22.919881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:22.920378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:22.923774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:22.987321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:23.022396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:23.023897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:23.036853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 999/1972] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:23.081252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:23.082276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:23.089572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:23.189461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd312d85-b7f6-4c1a-ac6b-4b17bf36d7e9 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:23.195792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:23.196931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:23.197325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:23.197708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] PATH_INFO: `/v3/projects/3b71e809a3b8407eb499c44f95264fdb` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:23.199036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b71e809a3b8407eb499c44f95264fdb)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:23.201315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:23.278489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b71e809a3b8407eb499c44f95264fdb', 'request_id': 'req-5caae624-ad03-4531-81a1-e7f2e72feafe'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.299169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b71e809a3b8407eb499c44f95264fdb', 'request_id': 'req-5caae624-ad03-4531-81a1-e7f2e72feafe'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.309481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b71e809a3b8407eb499c44f95264fdb', 'request_id': 'req-5caae624-ad03-4531-81a1-e7f2e72feafe'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.324212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] The token cache is being invalidate because project 3b71e809a3b8407eb499c44f95264fdb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:23.324505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.325321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.326929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 974/1973] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:23 2022] DELETE /identity/v3/projects/3b71e809a3b8407eb499c44f95264fdb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:23.331036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:23.353509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6770ab7-3ee8-4f08-a9fe-68db2952e8eb service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:23.455471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:23.457121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:23.457465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:23.458329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] PATH_INFO: `/v3/users/d99b7cc5aaf24d259cebe9c4fce361a0` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:23.460925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d99b7cc5aaf24d259cebe9c4fce361a0)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:23.463427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:23.531130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd99b7cc5aaf24d259cebe9c4fce361a0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.549671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd99b7cc5aaf24d259cebe9c4fce361a0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.561960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd99b7cc5aaf24d259cebe9c4fce361a0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.562809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd99b7cc5aaf24d259cebe9c4fce361a0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.563459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bba2d46-44ae-4048-9a22-5bdbd59749f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd99b7cc5aaf24d259cebe9c4fce361a0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:23.584796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1000/1974] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:23 2022] DELETE /identity/v3/users/d99b7cc5aaf24d259cebe9c4fce361a0 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:24.061296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.062791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:24.078735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caae624-ad03-4531-81a1-e7f2e72feafe admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.154071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.154938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.155231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.155620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] PATH_INFO: `/v3/projects/f083c3e502924f56b479c4e8421c26ea` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.156820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f083c3e502924f56b479c4e8421c26ea)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.158790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.216975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:24.231192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f083c3e502924f56b479c4e8421c26ea', 'request_id': 'req-e9221e82-4784-49ae-89be-db0fd774fc61'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:24.240802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f083c3e502924f56b479c4e8421c26ea', 'request_id': 'req-e9221e82-4784-49ae-89be-db0fd774fc61'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:24.250374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f083c3e502924f56b479c4e8421c26ea', 'request_id': 'req-e9221e82-4784-49ae-89be-db0fd774fc61'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:24.263963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] The token cache is being invalidate because project f083c3e502924f56b479c4e8421c26ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:24.264753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:24.267591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e9221e82-4784-49ae-89be-db0fd774fc61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:24.269763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 975/1975] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:24 2022] DELETE /identity/v3/projects/f083c3e502924f56b479c4e8421c26ea => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:24.274875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.275161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.275603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.294398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:24.296078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:24.364812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1001/1976] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:24.375557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.376029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.377197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.398399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:24.399699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:24.408897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 976/1977] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:24.413298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.425890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:24.432470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12b1e7-2c07-4d3f-a057-20b7e0f73612 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.439670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.440956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.441391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.441681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.443347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:24.443982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.445758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.463282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1002/1978] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:24.466570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.476434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c04b7-439f-4abb-ae07-c7108fdf8e79 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.484192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.484860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.485151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.485428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.486123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.488131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.522599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 977/1979] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:24 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:24.526457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.546283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7dca00-0c40-4e0e-b9dc-c7af0962ec06 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.613062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.613817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.614794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.615079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.615826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.617774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.640722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1003/1980] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:24 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:24.648517 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.659965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09ef16-fbd9-4e15-a3c5-6b88ceda9fb2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.668204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.668896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.669182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.669462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.672070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.675330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.688677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 978/1981] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:24.692857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.706055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc2300-1228-40a4-98f8-5b9882ffb8d5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.711455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.712175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.712628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.712924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.713757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.715531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.722496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.724972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:24.732451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1004/1982] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:24 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:24.736548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.748015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443ec4b2-8957-4f5b-8cd8-89f18115a2bd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.757563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8174afa-08d6-45d5-8491-93cdbc157666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.758768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8174afa-08d6-45d5-8491-93cdbc157666 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.759131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8174afa-08d6-45d5-8491-93cdbc157666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.759539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8174afa-08d6-45d5-8491-93cdbc157666 admin admin] PATH_INFO: `/v3/projects/72ee2a721f894d2494d05ebb610aab63/users/9dfcee38eb864d9d9fd04ff2f614c622/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.780386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8174afa-08d6-45d5-8491-93cdbc157666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ee2a721f894d2494d05ebb610aab63, user_id=9dfcee38eb864d9d9fd04ff2f614c622, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.784079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8174afa-08d6-45d5-8491-93cdbc157666 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.800468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1005/1983] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:24 2022] PUT /identity/v3/projects/72ee2a721f894d2494d05ebb610aab63/users/9dfcee38eb864d9d9fd04ff2f614c622/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:24.811252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.811556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.811968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.833376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540500f-afe6-4de5-ac48-32e5a016be04 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:24.857480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] MFA Rules not processed for user `9dfcee38eb864d9d9fd04ff2f614c622`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:24.858761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:24.904105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:24.904956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:24.905334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:24.905708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:24.908111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:24.968575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1006/1984] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:24.969661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:24.976547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:24.981929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:24.984175 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:24.997302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 979/1985] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:24 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:25.012667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:25.038223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0cf13b-a6bd-4595-98cf-7c185b6a52dc None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.043595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.044747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.044930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.045271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.046046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.048062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.084253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1007/1986] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:24 2022] POST /identity/v3/projects => generated 387 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:25.085739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.101815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7a8bd-5aa9-44ed-b618-ca652125d77a service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.186004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.186696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.186984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.187325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.188235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.190379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.213449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 980/1987] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:25 2022] POST /identity/v3/users => generated 300 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:25.217986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.231620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ef49ff-eeb0-43dc-bb1b-0f983e4c480f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.239553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa12abce-3b04-4882-9993-2eaff19db810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.240454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12abce-3b04-4882-9993-2eaff19db810 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.240733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12abce-3b04-4882-9993-2eaff19db810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.241010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa12abce-3b04-4882-9993-2eaff19db810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.241744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12abce-3b04-4882-9993-2eaff19db810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.243444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa12abce-3b04-4882-9993-2eaff19db810 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.265889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1008/1988] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:25.270686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.282601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d047a2-008c-4abd-bcc1-fd4a31ec61e4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.288584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.289542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.289881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.290127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] PATH_INFO: `/v3/projects/e8a6dbe8156f4283ac621096b8496cd5/users/95c69e3398144831a99ea7a84e383317/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.312727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a7e78-dea6-4ae6-bbc9-94b381ddc282 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.313278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a7e78-dea6-4ae6-bbc9-94b381ddc282 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.313543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a6dbe8156f4283ac621096b8496cd5, user_id=95c69e3398144831a99ea7a84e383317, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.313926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a7e78-dea6-4ae6-bbc9-94b381ddc282 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.315921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1009/1989] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:42:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 24 08:42:25.316309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.327864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.328293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.328679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.334939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 981/1990] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:25 2022] PUT /identity/v3/projects/e8a6dbe8156f4283ac621096b8496cd5/users/95c69e3398144831a99ea7a84e383317/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:25.339092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.354147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] MFA Rules not processed for user `a39a333f4c42420cb46189d32aafc6fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:25.355608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:25.440454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1010/1991] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:42:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:25.443467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f56a1b-1796-4236-8ccf-c2cb7a99fe66 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.449090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.449873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.450243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.450594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.451449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.453146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.464490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 982/1992] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:25 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:25.468563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.475797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.483597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb84a27-0512-4f60-83ba-60ab64ec5286 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.485092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:25.498911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69025d-d717-47e5-8f99-3279d3e6204c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.499849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69025d-d717-47e5-8f99-3279d3e6204c admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.500131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69025d-d717-47e5-8f99-3279d3e6204c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.500408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69025d-d717-47e5-8f99-3279d3e6204c admin admin] PATH_INFO: `/v3/projects/e8a6dbe8156f4283ac621096b8496cd5/users/95c69e3398144831a99ea7a84e383317/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.504179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69025d-d717-47e5-8f99-3279d3e6204c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8a6dbe8156f4283ac621096b8496cd5, user_id=95c69e3398144831a99ea7a84e383317, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.506885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69025d-d717-47e5-8f99-3279d3e6204c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.515466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76dc4b3-6228-4461-89b1-6600f0b52564 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.527301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1011/1993] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:25 2022] PUT /identity/v3/projects/e8a6dbe8156f4283ac621096b8496cd5/users/95c69e3398144831a99ea7a84e383317/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:25.536404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.537089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.537374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.537547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.537657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.538068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.538599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.548383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.550686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.566946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 983/1994] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:25.584621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] MFA Rules not processed for user `95c69e3398144831a99ea7a84e383317`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:25.586789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:25.695189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1012/1995] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:25.706970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.709006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:25.759446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:25.772654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bbb922-802f-44ed-876d-ce1eff2d015a service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.821852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.822510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.822793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.823101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.832212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.834183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:25.843794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 984/1996] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:25.872671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:25.893590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:25.958997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0152df49-faff-4cb9-be2f-bf97b38f3189 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:25.964966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:25.966188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:25.966724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:25.967146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] PATH_INFO: `/v3/users/9dfcee38eb864d9d9fd04ff2f614c622` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:25.970581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfcee38eb864d9d9fd04ff2f614c622)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:25.974732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.018596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.020198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.041311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97e1d1-5754-4183-a80b-eafcd10ecc89 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:26.047912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:26.049210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.049338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.049700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] PATH_INFO: `/v3/users/40dac9dfb79d45d2a84d581698e5f663` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.050301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfcee38eb864d9d9fd04ff2f614c622'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.051287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40dac9dfb79d45d2a84d581698e5f663)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:26.053943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.061309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfcee38eb864d9d9fd04ff2f614c622'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.073696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfcee38eb864d9d9fd04ff2f614c622'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.076240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfcee38eb864d9d9fd04ff2f614c622'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.077013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-2fc421a1-2414-4142-8dea-599015e2a119 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfcee38eb864d9d9fd04ff2f614c622'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.102130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1013/1997] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:25 2022] DELETE /identity/v3/users/9dfcee38eb864d9d9fd04ff2f614c622 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:26.106737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.107238 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.107471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.117492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40dac9dfb79d45d2a84d581698e5f663'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.124747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:26.126054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40dac9dfb79d45d2a84d581698e5f663'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.126192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.142884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40dac9dfb79d45d2a84d581698e5f663'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.151739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40dac9dfb79d45d2a84d581698e5f663'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.152458 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40dac9dfb79d45d2a84d581698e5f663'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.163483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 985/1998] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:26 2022] DELETE /identity/v3/users/40dac9dfb79d45d2a84d581698e5f663 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:26.202864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1014/1999] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:26.213946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.215432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.262095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.263554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.285307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ddf2a-b7e7-48fc-9a2a-6e71a2b54180 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:26.289604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:26.290263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.290547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.291080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.339946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:26.340945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:26.381705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d5f2d4-3394-4d05-a0b9-785794cd16dd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:26.400793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:26.402432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.414037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 986/2000] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:26.414801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.416550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.429522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd14760-959e-40df-86d2-b6522a92a90f service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:26.434146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:26.435008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.435321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.435896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] PATH_INFO: `/v3/projects/3fd1e42945794a1096d0269bb1656482` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.437076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fd1e42945794a1096d0269bb1656482)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:26.439470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.449492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:26.450152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.450446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.450758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.452953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.497399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fd1e42945794a1096d0269bb1656482', 'request_id': 'req-c45624c6-670b-4b00-960f-c109ef31616c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.522578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fd1e42945794a1096d0269bb1656482', 'request_id': 'req-c45624c6-670b-4b00-960f-c109ef31616c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.532820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fd1e42945794a1096d0269bb1656482', 'request_id': 'req-c45624c6-670b-4b00-960f-c109ef31616c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.545218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:26.546053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] The token cache is being invalidate because project 3fd1e42945794a1096d0269bb1656482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:26.546353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.547067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.547179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.548901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 987/2001] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:26 2022] DELETE /identity/v3/projects/3fd1e42945794a1096d0269bb1656482 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:26.553160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.554704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.563347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1015/2002] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:26.633123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45624c6-670b-4b00-960f-c109ef31616c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:26.639601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:26.640488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.640771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.641046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] PATH_INFO: `/v3/users/0311fbfbf7d64b1e9eebffa3f3b9600d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.642379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0311fbfbf7d64b1e9eebffa3f3b9600d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:26.644425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.709608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0311fbfbf7d64b1e9eebffa3f3b9600d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.711544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:26.720833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0311fbfbf7d64b1e9eebffa3f3b9600d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.736977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0311fbfbf7d64b1e9eebffa3f3b9600d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.744804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0311fbfbf7d64b1e9eebffa3f3b9600d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.746058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0311fbfbf7d64b1e9eebffa3f3b9600d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.753180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 988/2003] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:26 2022] DELETE /identity/v3/users/0311fbfbf7d64b1e9eebffa3f3b9600d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:26.797726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.799175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.813886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c9b8b-a4ee-44dc-809c-9fd902f4ea9c service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:26.902408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:26.903492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:26.904542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:26.904542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] PATH_INFO: `/v3/projects/72ee2a721f894d2494d05ebb610aab63` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:26.905691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ee2a721f894d2494d05ebb610aab63)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:26.908361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:26.969328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:26.977879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:26.979379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ee2a721f894d2494d05ebb610aab63', 'request_id': 'req-ecb013ae-c461-4dcb-a96a-706f74ec049c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:26.979694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:26.993738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ee2a721f894d2494d05ebb610aab63', 'request_id': 'req-ecb013ae-c461-4dcb-a96a-706f74ec049c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.002184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ee2a721f894d2494d05ebb610aab63', 'request_id': 'req-ecb013ae-c461-4dcb-a96a-706f74ec049c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.002828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a477b-e9ee-460b-878c-9e76f746ea9c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.032534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] The token cache is being invalidate because project 72ee2a721f894d2494d05ebb610aab63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:27.032848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.033537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.038693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1016/2004] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:26 2022] DELETE /identity/v3/projects/72ee2a721f894d2494d05ebb610aab63 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:27.040346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.042499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.084290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.085676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.085958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.086253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] PATH_INFO: `/v3/projects/4b45cba40d96425ab7e1cca987fdeb71` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.088151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b45cba40d96425ab7e1cca987fdeb71)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.089697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.126556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb013ae-c461-4dcb-a96a-706f74ec049c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.132223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.133178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.133464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.133813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] PATH_INFO: `/v3/users/95c69e3398144831a99ea7a84e383317` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.135184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c69e3398144831a99ea7a84e383317)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.137491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.157890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b45cba40d96425ab7e1cca987fdeb71', 'request_id': 'req-b0d9fade-2115-4e5d-bd57-0959755a3407'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.168355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b45cba40d96425ab7e1cca987fdeb71', 'request_id': 'req-b0d9fade-2115-4e5d-bd57-0959755a3407'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.177867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b45cba40d96425ab7e1cca987fdeb71', 'request_id': 'req-b0d9fade-2115-4e5d-bd57-0959755a3407'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.192073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] The token cache is being invalidate because project 4b45cba40d96425ab7e1cca987fdeb71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:27.192502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.193357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b0d9fade-2115-4e5d-bd57-0959755a3407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.195202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 989/2005] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:26 2022] DELETE /identity/v3/projects/4b45cba40d96425ab7e1cca987fdeb71 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:27.197891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c69e3398144831a99ea7a84e383317'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.200126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.200395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.200673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.214429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c69e3398144831a99ea7a84e383317'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.221700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:27.223140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.225144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c69e3398144831a99ea7a84e383317'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.225773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c69e3398144831a99ea7a84e383317'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.226404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6fd98b3a-5f7a-45d3-ad4b-b66197e3ccc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c69e3398144831a99ea7a84e383317'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.248431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1017/2006] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:27 2022] DELETE /identity/v3/users/95c69e3398144831a99ea7a84e383317 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:27.277154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:27.301869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 990/2007] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:27.310130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.310498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.310784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.330642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:27.331908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.382843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:27.400033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1018/2008] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:27.405583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.416336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e696e032-c09c-4470-95a2-344b34fcaced None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.420974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.421657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.421941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.422217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.422947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:27.423279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.424871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.437853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 991/2009] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:27.442691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.452216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff060e80-cf7b-47d0-8dd5-4b8a57005a7c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.459904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.460684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.461068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.461447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.462294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.464452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.495407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1019/2010] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:27.496794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.508064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ca827e-e3e5-4833-bd84-a97200ec16eb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.563983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.565948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.587946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.589096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.590118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.590411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.591307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.593959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.619646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 992/2011] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:27.624254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.638580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab2a452-3ca5-4ca1-8f3b-6ad1c7191f88 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.643131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e13c1fd-2f1e-4632-834f-4e81085d4bf2 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.648875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.649760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.650045 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.650324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] PATH_INFO: `/v3/projects/e8a6dbe8156f4283ac621096b8496cd5` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.651444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.651586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8a6dbe8156f4283ac621096b8496cd5)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.653457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.655463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.655862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.656662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.657525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.659319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.671496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 993/2012] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:27.675390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.688583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce001a20-45c0-47c3-988c-8eec3ca18efe admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.694182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.695403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.695848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.696289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.697267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.699271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.712393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 994/2013] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:27.716197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.729659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8a6dbe8156f4283ac621096b8496cd5', 'request_id': 'req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.730860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c768a61-dc31-40cb-a9a5-a06b8a7dde8d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.735751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8a6dbe8156f4283ac621096b8496cd5', 'request_id': 'req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.742224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8a6dbe8156f4283ac621096b8496cd5', 'request_id': 'req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.755556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] The token cache is being invalidate because project e8a6dbe8156f4283ac621096b8496cd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:27.755939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.756645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ae0b071-bd64-43ae-9c1b-dd1b0837f047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:27.758552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1020/2014] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:27 2022] DELETE /identity/v3/projects/e8a6dbe8156f4283ac621096b8496cd5 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:27.763112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd30667-c616-427c-93a5-d434bd284dc6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.763432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd30667-c616-427c-93a5-d434bd284dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.763715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd30667-c616-427c-93a5-d434bd284dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.782922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5bd30667-c616-427c-93a5-d434bd284dc6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:27.784207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5bd30667-c616-427c-93a5-d434bd284dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.801439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82128c-700c-469a-954f-a18c229a884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.802944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82128c-700c-469a-954f-a18c229a884d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.803501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82128c-700c-469a-954f-a18c229a884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.803988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82128c-700c-469a-954f-a18c229a884d admin admin] PATH_INFO: `/v3/projects/3487ca3da7d1412fbaf0f999310ffda1/users/192a573e15a841b4b40e682f1754623e/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.820709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82128c-700c-469a-954f-a18c229a884d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3487ca3da7d1412fbaf0f999310ffda1, user_id=192a573e15a841b4b40e682f1754623e, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.823432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82128c-700c-469a-954f-a18c229a884d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.836685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 995/2015] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:27 2022] PUT /identity/v3/projects/3487ca3da7d1412fbaf0f999310ffda1/users/192a573e15a841b4b40e682f1754623e/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:27.847642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.848032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.848345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.848976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1021/2016] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:27.857491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.857865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.858261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.865317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:27.874920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] MFA Rules not processed for user `192a573e15a841b4b40e682f1754623e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:27.876167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.880091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:27.881256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.895867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:27.941048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 996/2017] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:27.945540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.946936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:27.948182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1022/2018] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:27.952043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:27.961822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8379f8-2360-46ed-b26f-bffe28d909f0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.962130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18f75a8-80a7-4fd3-9e3c-0bb3abd59299 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:27.966611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.967301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.967474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:27.967598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.967846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.968185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:27.968475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:27.968637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:27.968753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:27.968952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.969431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:27.970508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.971527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:27.984091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1023/2019] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:27.989032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.003413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 997/2020] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:28.003621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b0ba66-f292-429a-ad25-87bff750418b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.004882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.073170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732504c6-2f3e-450a-95df-3dd52ce8f15e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.077937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.078590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.078957 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.079256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.079953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.082241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.083382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.084183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.084578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.084927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.085698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.087738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.107251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 998/2021] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:28 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:28.108252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.118783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1024/2022] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:27 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:28.122762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.122892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd4469e-4547-41ab-9913-810e066cd061 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.202514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.202704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f21c-ba95-4c5f-a615-2b05f1948101 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.202802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.202802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.202802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.202802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.202802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.203180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.203180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.203384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.203709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.204587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.206732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.215539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 999/2023] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:28.216583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.228332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1a85-2580-45f9-a9d5-b3c892a2e91b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.233826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1025/2024] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:28 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:28.234200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.235436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.235855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.236192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] PATH_INFO: `/v3/projects/53a0a3b1d78140d59e8c6151e86761d3/users/cc40dae94ab6479886765225f77aa8d6/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.237985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.245207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:28.249231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecb408-556c-4d08-8447-d3ccf5949fa8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.253816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a0a3b1d78140d59e8c6151e86761d3, user_id=cc40dae94ab6479886765225f77aa8d6, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.255395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.256063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.256344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.256624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.257268 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.257368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.260321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.271905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1000/2025] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:28 2022] PUT /identity/v3/projects/53a0a3b1d78140d59e8c6151e86761d3/users/cc40dae94ab6479886765225f77aa8d6/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:28.272138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1026/2026] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:28.275962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.278353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.280524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:28.288443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d687c6-dcb6-43b8-a5c1-5ff214532e65 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.343181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f2a73-cc03-4a39-ab47-66fd4597b79d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.347969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.348687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.348967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.349242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.349968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.351591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.356910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.357669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.358025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.358372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.359174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.361113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.362615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1001/2027] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:28.366692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.375708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1027/2028] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:28.377180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dcfb72-29a6-4f74-b784-a06833c665e5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.380153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.383000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce65fae-e254-44ef-b844-ee88358f84e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.384765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65fae-e254-44ef-b844-ee88358f84e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.384765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65fae-e254-44ef-b844-ee88358f84e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.384765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce65fae-e254-44ef-b844-ee88358f84e2 admin admin] PATH_INFO: `/v3/projects/fd236d93f3124ec88e0bc1a91cdb8bf0/users/5f6afedd91c1445db4404a746a48968c/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.390596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bbe872-43d2-47df-b780-5553d4ceb17b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.402731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce65fae-e254-44ef-b844-ee88358f84e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd236d93f3124ec88e0bc1a91cdb8bf0, user_id=5f6afedd91c1445db4404a746a48968c, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.406534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce65fae-e254-44ef-b844-ee88358f84e2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.407599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522d7ea-de6b-4d3b-b213-e9b54cae2128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.408586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522d7ea-de6b-4d3b-b213-e9b54cae2128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.408939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522d7ea-de6b-4d3b-b213-e9b54cae2128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.409290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522d7ea-de6b-4d3b-b213-e9b54cae2128 admin admin] PATH_INFO: `/v3/projects/53a0a3b1d78140d59e8c6151e86761d3/users/cc40dae94ab6479886765225f77aa8d6/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.411752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522d7ea-de6b-4d3b-b213-e9b54cae2128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53a0a3b1d78140d59e8c6151e86761d3, user_id=cc40dae94ab6479886765225f77aa8d6, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.415940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522d7ea-de6b-4d3b-b213-e9b54cae2128 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.424348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1002/2029] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:28 2022] PUT /identity/v3/projects/fd236d93f3124ec88e0bc1a91cdb8bf0/users/5f6afedd91c1445db4404a746a48968c/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:28.429226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1028/2030] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:28 2022] PUT /identity/v3/projects/53a0a3b1d78140d59e8c6151e86761d3/users/cc40dae94ab6479886765225f77aa8d6/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:28.437673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.437673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.437673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.439363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.439744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.440092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.469604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] MFA Rules not processed for user `5f6afedd91c1445db4404a746a48968c`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:28.471021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:28.472807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] MFA Rules not processed for user `cc40dae94ab6479886765225f77aa8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:28.474129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:28.548367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1003/2031] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:28 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:28.549048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1029/2032] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:28.553488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.562502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.564124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:28.615646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78eb7f97-0fdf-4a87-b68b-a12ab4ed1103 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.620185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.620846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.621149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.621404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.621667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4904d086-ff3d-43f8-96b6-d2848415fbce None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.622110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.624117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.625515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.626268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.626444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.626721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.633296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:28.640626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:28.653042 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1004/2033] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:28 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:28.656955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.692536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.694094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.703972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1030/2034] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:28.723390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef57bc11-854e-4bd2-b353-285ca75b3b67 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.728549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.729220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.729502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.729781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.730480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.732448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.758671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1005/2035] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:28 2022] POST /identity/v3/users => generated 307 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:28.762858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.789142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0afb3a-3195-41cf-83f9-613b489845e5 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.794787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.795740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.796095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.796441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.797236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.798990 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.814796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1031/2036] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:28.816665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.834188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ab09c9-972b-46c6-9e76-f0d529cf32c4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.839685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.841113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.841686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.842095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] PATH_INFO: `/v3/projects/b9b82a8f884f44e4bb120e00b04a3196/users/fe06ffa88c11471a84b12a12bd66e0a9/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.868384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b82a8f884f44e4bb120e00b04a3196, user_id=fe06ffa88c11471a84b12a12bd66e0a9, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.871896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:28.893526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1006/2037] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:28 2022] PUT /identity/v3/projects/b9b82a8f884f44e4bb120e00b04a3196/users/fe06ffa88c11471a84b12a12bd66e0a9/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:28.898217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:28.915250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3daa041-bd3f-451a-9c59-7efd0182c036 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:28.993040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f1ecf-8bb9-4ee1-b5d6-54ba962bfed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:28.993720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f1ecf-8bb9-4ee1-b5d6-54ba962bfed8 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:28.994009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f1ecf-8bb9-4ee1-b5d6-54ba962bfed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:28.994285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f1ecf-8bb9-4ee1-b5d6-54ba962bfed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:28.995013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f1ecf-8bb9-4ee1-b5d6-54ba962bfed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:28.996645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f1ecf-8bb9-4ee1-b5d6-54ba962bfed8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:29.008590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1032/2038] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:28 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:29.013330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:29.026902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a288163-e80e-4770-acb1-4c1b57141350 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:29.033305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:29.034209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.034661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.035410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] PATH_INFO: `/v3/projects/b9b82a8f884f44e4bb120e00b04a3196/users/fe06ffa88c11471a84b12a12bd66e0a9/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.038358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9b82a8f884f44e4bb120e00b04a3196, user_id=fe06ffa88c11471a84b12a12bd66e0a9, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:29.041768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:29.057913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1007/2039] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:29 2022] PUT /identity/v3/projects/b9b82a8f884f44e4bb120e00b04a3196/users/fe06ffa88c11471a84b12a12bd66e0a9/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:29.069262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.069625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.070014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.103601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] MFA Rules not processed for user `fe06ffa88c11471a84b12a12bd66e0a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:29.105503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:29.189884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1033/2040] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:29 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:29.201286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:29.203011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:29.268206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68989558-fa64-4add-81eb-88d43d4c6cbf admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:29.272689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:29.273384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.273674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.274096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.286785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:29.288299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:29.298578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1008/2041] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:29 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:29.370618 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:29.380682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249d0b6-410f-411e-8afb-ec35019cb013 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:29.384340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9cca06-fef3-450d-a67d-d6ebb3f836d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:29.384994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cca06-fef3-450d-a67d-d6ebb3f836d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:29.385277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cca06-fef3-450d-a67d-d6ebb3f836d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:29.385557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9cca06-fef3-450d-a67d-d6ebb3f836d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:29.412524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:29.452042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cca06-fef3-450d-a67d-d6ebb3f836d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:29.453711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9cca06-fef3-450d-a67d-d6ebb3f836d5 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:29.467450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1034/2042] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:29 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:35.845894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:35.847340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:35.953530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:36.006906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7f512-d47a-4c53-ab1b-26d79116de3f service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:36.011804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:36.012699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:36.012984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:36.013471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] PATH_INFO: `/v3/users/aff37b3f7c864aa0b7f95626c57f185d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:36.014810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=aff37b3f7c864aa0b7f95626c57f185d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:36.016812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:36.106529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aff37b3f7c864aa0b7f95626c57f185d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.117417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aff37b3f7c864aa0b7f95626c57f185d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.137835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aff37b3f7c864aa0b7f95626c57f185d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.147092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aff37b3f7c864aa0b7f95626c57f185d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.148808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aff37b3f7c864aa0b7f95626c57f185d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.158677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1009/2043] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:35 2022] DELETE /identity/v3/users/aff37b3f7c864aa0b7f95626c57f185d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:36.163132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:36.163574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:36.163723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:36.190860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:36.192180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:36.261442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1035/2044] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:36.271657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:36.273036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:36.338305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df5b643-8c71-4fd5-b45a-716b250f3ddc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:36.342207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:36.343067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:36.343467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:36.343806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:36.357954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:36.359488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:36.372119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1010/2045] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:36.793890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:36.795692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:36.814305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930dc8d-9d38-4ddc-91e8-7dadafab7e5b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:36.823993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:36.825051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:36.825444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:36.825854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] PATH_INFO: `/v3/projects/dfdad00feabb44c6b95fa0d5b328fb5f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:36.827347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdad00feabb44c6b95fa0d5b328fb5f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:36.829631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:36.917133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdad00feabb44c6b95fa0d5b328fb5f', 'request_id': 'req-a7dcb448-bfcc-4738-ab98-0289bf350546'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.924482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdad00feabb44c6b95fa0d5b328fb5f', 'request_id': 'req-a7dcb448-bfcc-4738-ab98-0289bf350546'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.932188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdad00feabb44c6b95fa0d5b328fb5f', 'request_id': 'req-a7dcb448-bfcc-4738-ab98-0289bf350546'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.948248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] The token cache is being invalidate because project dfdad00feabb44c6b95fa0d5b328fb5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:36.948563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.949287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:36.950904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1036/2046] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:36 2022] DELETE /identity/v3/projects/dfdad00feabb44c6b95fa0d5b328fb5f => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:36.954896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:36.966480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f1964d-33c8-4bf8-94d5-47f8f6d6d0b6 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:37.076872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:37.077831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:37.078186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:37.078551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] PATH_INFO: `/v3/users/97e6608cd59e42d390bfd95eb91396cb` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:37.079973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97e6608cd59e42d390bfd95eb91396cb)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:37.082020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:37.091287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:37.153724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97e6608cd59e42d390bfd95eb91396cb'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.164805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97e6608cd59e42d390bfd95eb91396cb'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.189594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97e6608cd59e42d390bfd95eb91396cb'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.198887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97e6608cd59e42d390bfd95eb91396cb'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.199734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97e6608cd59e42d390bfd95eb91396cb'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.209003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1011/2047] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:36 2022] DELETE /identity/v3/users/97e6608cd59e42d390bfd95eb91396cb => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:37.648364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:37.650124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:37.669080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dcb448-bfcc-4738-ab98-0289bf350546 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:37.719659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:37.738511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:37.739446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:37.739730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:37.740007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] PATH_INFO: `/v3/projects/dbef7dad1b564f0eabd5f711608429bb` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:37.741258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbef7dad1b564f0eabd5f711608429bb)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:37.743238 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:37.778338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:37.779929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:37.793724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a55591-4fa8-487a-8025-887d3ff60086 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:37.793890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbef7dad1b564f0eabd5f711608429bb', 'request_id': 'req-6912652b-664a-4b8c-8572-012e1074bc81'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.799375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbef7dad1b564f0eabd5f711608429bb', 'request_id': 'req-6912652b-664a-4b8c-8572-012e1074bc81'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.806855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbef7dad1b564f0eabd5f711608429bb', 'request_id': 'req-6912652b-664a-4b8c-8572-012e1074bc81'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.820813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] The token cache is being invalidate because project dbef7dad1b564f0eabd5f711608429bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:37.821205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.821891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.823488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1037/2048] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:37 2022] DELETE /identity/v3/projects/dbef7dad1b564f0eabd5f711608429bb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:37.827621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:37.829006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:37.860414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:37.861382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:37.861670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:37.861990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] PATH_INFO: `/v3/users/5f6afedd91c1445db4404a746a48968c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:37.863359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f6afedd91c1445db4404a746a48968c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:37.865955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:37.910312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6912652b-664a-4b8c-8572-012e1074bc81 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:37.915788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:37.916198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:37.916456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:37.916734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] PATH_INFO: `/v3/users/502dec5d05aa477e8c84b7a3f00005f6` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:37.917949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=502dec5d05aa477e8c84b7a3f00005f6)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:37.919913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:37.925772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f6afedd91c1445db4404a746a48968c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.933763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f6afedd91c1445db4404a746a48968c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.948113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f6afedd91c1445db4404a746a48968c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.955568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f6afedd91c1445db4404a746a48968c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.956355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f29a6a4f-fff4-42be-8ff7-0fdeaaccf3ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f6afedd91c1445db4404a746a48968c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.962443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1012/2049] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:37 2022] DELETE /identity/v3/users/5f6afedd91c1445db4404a746a48968c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:37.966662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:37.967146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:37.967496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:37.971137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '502dec5d05aa477e8c84b7a3f00005f6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.974245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:37.978788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '502dec5d05aa477e8c84b7a3f00005f6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.985813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:37.986511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '502dec5d05aa477e8c84b7a3f00005f6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.987437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '502dec5d05aa477e8c84b7a3f00005f6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:37.987854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:37.988083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '502dec5d05aa477e8c84b7a3f00005f6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.002189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:38.008560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1038/2050] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:37 2022] DELETE /identity/v3/users/502dec5d05aa477e8c84b7a3f00005f6 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:38.066697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1013/2051] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:38.075565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:38.077127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:38.151830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d9aaf-9e45-479b-adb3-d9a7915732a0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:38.155466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:38.156115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:38.156396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:38.156670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:38.225151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:38.226698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:38.238090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1039/2052] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:38.615644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:38.617278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:38.645236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea5dab3-a414-4168-a48d-c63a91699696 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:38.662876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:38.664221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:38.664701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:38.665023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] PATH_INFO: `/v3/projects/fd236d93f3124ec88e0bc1a91cdb8bf0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:38.666947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd236d93f3124ec88e0bc1a91cdb8bf0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:38.670283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:38.672852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:38.685354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:38.696448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e8f839-458a-461f-b549-751f838a15d6 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:38.710733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:38.712260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:38.713704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:38.714305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] PATH_INFO: `/v3/projects/d18f3acf3e9c4e46964b14bede4ad06a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:38.716907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d18f3acf3e9c4e46964b14bede4ad06a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:38.720548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:38.764699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd236d93f3124ec88e0bc1a91cdb8bf0', 'request_id': 'req-3b0db8b4-efb7-4b88-b829-917a13eec8a1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.779946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd236d93f3124ec88e0bc1a91cdb8bf0', 'request_id': 'req-3b0db8b4-efb7-4b88-b829-917a13eec8a1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.793004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd236d93f3124ec88e0bc1a91cdb8bf0', 'request_id': 'req-3b0db8b4-efb7-4b88-b829-917a13eec8a1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.795997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd18f3acf3e9c4e46964b14bede4ad06a', 'request_id': 'req-f1967520-692b-4e1b-ad98-594a41187e6b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.802484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd18f3acf3e9c4e46964b14bede4ad06a', 'request_id': 'req-f1967520-692b-4e1b-ad98-594a41187e6b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.810309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] The token cache is being invalidate because project fd236d93f3124ec88e0bc1a91cdb8bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:38.810309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.811011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.811485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd18f3acf3e9c4e46964b14bede4ad06a', 'request_id': 'req-f1967520-692b-4e1b-ad98-594a41187e6b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.817286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1014/2053] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:38 2022] DELETE /identity/v3/projects/fd236d93f3124ec88e0bc1a91cdb8bf0 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:38.817286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:38.818459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:38.831935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] The token cache is being invalidate because project d18f3acf3e9c4e46964b14bede4ad06a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:38.832509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.833280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:38.836822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1040/2054] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:38 2022] DELETE /identity/v3/projects/d18f3acf3e9c4e46964b14bede4ad06a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:38.837975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:38.842481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:38.940111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:38.972303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0db8b4-efb7-4b88-b829-917a13eec8a1 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.047180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1967520-692b-4e1b-ad98-594a41187e6b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.048479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:39.059525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.060460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.060819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.061170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.063535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:39.069148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.070052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.070437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.070592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] PATH_INFO: `/v3/users/fe06ffa88c11471a84b12a12bd66e0a9` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.072444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe06ffa88c11471a84b12a12bd66e0a9)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.074386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.178998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.180951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17358491-0aba-45f5-ac92-441ee79f1af9 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.198572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe06ffa88c11471a84b12a12bd66e0a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.207178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1015/2055] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:39.209186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042d584-b594-4472-97b5-cdcc7ee7cb2d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.209822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042d584-b594-4472-97b5-cdcc7ee7cb2d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.210202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2042d584-b594-4472-97b5-cdcc7ee7cb2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.215464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe06ffa88c11471a84b12a12bd66e0a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.236968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe06ffa88c11471a84b12a12bd66e0a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.240470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe06ffa88c11471a84b12a12bd66e0a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.241788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe06ffa88c11471a84b12a12bd66e0a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.245252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2042d584-b594-4472-97b5-cdcc7ee7cb2d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:39.246823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2042d584-b594-4472-97b5-cdcc7ee7cb2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:39.263901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1016/2056] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:39.265365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.265655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.265936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.289329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1041/2057] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:38 2022] DELETE /identity/v3/users/fe06ffa88c11471a84b12a12bd66e0a9 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:39.297900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:39.299500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:39.403105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1017/2058] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:39.404230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.418012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a108a2-c808-40f1-b75b-fc2aafdafec9 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.429390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.432589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.432761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.434189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.437290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:39.437762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.450011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.481830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1042/2059] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:39.485918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.497724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9786bd26-ecbc-4e8e-ab9d-b9e896ef3d33 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.503643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.504949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.505303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.505705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.506457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.508801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.545928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1018/2060] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:39 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:39.550859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.563307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a357a96-deb3-4674-abed-b4cd03cc0e1d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.642957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.644109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.644395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.644672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.645598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.647839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.677968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1043/2061] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:39 2022] POST /identity/v3/users => generated 287 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:39.681755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.694709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8280ea-8929-4fa2-a1c2-7efca20460fc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.699711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.700884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.700884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.701076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.701664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.703855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.717040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1019/2062] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:39.718222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.729789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1edfee1-8940-4d60-9f04-df25397719bf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.734641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.735350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.735636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.735912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.737241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.738316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.750246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1044/2063] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:39.754079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.764831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94c601d-3be7-4ec5-aefb-b1c3bf1c7427 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.770925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7760765-4b15-4411-87f4-c6e83d38a326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.771961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760765-4b15-4411-87f4-c6e83d38a326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.772310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760765-4b15-4411-87f4-c6e83d38a326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.772658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7760765-4b15-4411-87f4-c6e83d38a326 admin admin] PATH_INFO: `/v3/projects/8b4f777ff73849638c19d79a4e38cb49/users/b2e868e843fa42f89c9bfe49459531a9/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.791347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7760765-4b15-4411-87f4-c6e83d38a326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b4f777ff73849638c19d79a4e38cb49, user_id=b2e868e843fa42f89c9bfe49459531a9, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.794620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7760765-4b15-4411-87f4-c6e83d38a326 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.815353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.815353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:39.815539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1020/2064] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:39 2022] PUT /identity/v3/projects/8b4f777ff73849638c19d79a4e38cb49/users/b2e868e843fa42f89c9bfe49459531a9/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:39.821281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.821646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.821939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.823945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8326b-6fbe-4595-8948-c0e6a694a4e3 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:39.844441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:39.854826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] MFA Rules not processed for user `b2e868e843fa42f89c9bfe49459531a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:39.856234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:39.875308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:39.899920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:39.900796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.901088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.901367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] PATH_INFO: `/v3/projects/b9b82a8f884f44e4bb120e00b04a3196` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:39.902721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9b82a8f884f44e4bb120e00b04a3196)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:39.904693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:39.926183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1021/2065] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:39.936883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:39.938469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:39.954173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9b82a8f884f44e4bb120e00b04a3196', 'request_id': 'req-693a6975-7bb4-4701-8dc2-2b2b58d6a306'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.960161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9b82a8f884f44e4bb120e00b04a3196', 'request_id': 'req-693a6975-7bb4-4701-8dc2-2b2b58d6a306'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.967595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9b82a8f884f44e4bb120e00b04a3196', 'request_id': 'req-693a6975-7bb4-4701-8dc2-2b2b58d6a306'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.983773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] The token cache is being invalidate because project b9b82a8f884f44e4bb120e00b04a3196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:39.984118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.984908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-693a6975-7bb4-4701-8dc2-2b2b58d6a306 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:39.991482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1045/2066] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:39 2022] DELETE /identity/v3/projects/b9b82a8f884f44e4bb120e00b04a3196 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:39.993028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb12f27-d63e-4302-ac51-fc76f8eb4a2e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:39.993324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb12f27-d63e-4302-ac51-fc76f8eb4a2e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:39.993610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb12f27-d63e-4302-ac51-fc76f8eb4a2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.016477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4624138-8e82-415f-86c4-439f0dd6d99a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.018409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8bb12f27-d63e-4302-ac51-fc76f8eb4a2e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:40.019882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8bb12f27-d63e-4302-ac51-fc76f8eb4a2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:40.077112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.077834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.079005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.079444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.081428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:40.133400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1046/2067] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:40.135547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.136217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.136746 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.158058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.159591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.161429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:40.162838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:40.178742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1047/2068] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:40.182705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1022/2069] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:39 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:40.185076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.195562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f9578-03cf-447d-83f8-62cabbb6efa1 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.202772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.203686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.204151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.204604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.205522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:40.206025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.208173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.222445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1048/2070] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:40.227468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.238493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee5718-fe6f-499a-b108-fff29e33ede2 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.244075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.244748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.245036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.245559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.246008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.247965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.282704 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1023/2071] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:40 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:40.283890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.294627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c85972-fce0-4774-81fc-0b246018e6fb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.320168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:40.356973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.357656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.357944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.358225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.358970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.360955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.382682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1049/2072] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:40 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:40.386831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.397258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f2c2b-935f-4fc8-9659-ed6d5ca5b1d3 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.402216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.402968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.403288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.403609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.404385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.406087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.408307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:40.419245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1024/2073] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:40.422847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.433297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8be5ab-c8e9-4f91-88dd-273b2dea632c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.438539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422e21c-3fe7-4376-a9cc-6479649c951f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.439233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422e21c-3fe7-4376-a9cc-6479649c951f admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.439535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422e21c-3fe7-4376-a9cc-6479649c951f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.439816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422e21c-3fe7-4376-a9cc-6479649c951f admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.440549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422e21c-3fe7-4376-a9cc-6479649c951f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.442671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3422e21c-3fe7-4376-a9cc-6479649c951f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.456362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1050/2074] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:40.458113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.479775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef288c5-6016-4930-9a93-45377ccde424 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.485061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.485971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.486255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.486533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] PATH_INFO: `/v3/projects/e7e9685d896948a3a44c70a91459306d/users/47238795697740a290a2fb24e14e4282/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.505169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7e9685d896948a3a44c70a91459306d, user_id=47238795697740a290a2fb24e14e4282, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.509519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.555379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1025/2075] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:40 2022] PUT /identity/v3/projects/e7e9685d896948a3a44c70a91459306d/users/47238795697740a290a2fb24e14e4282/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:40.565209 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.565505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.565791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.609615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] MFA Rules not processed for user `47238795697740a290a2fb24e14e4282`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:40.610952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:40.716408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1051/2076] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:40.724390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:40.729860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:40.802455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aebbf9-1c9d-4c4b-bf02-7a2554f79bd2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:40.807060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee8b97-a255-4735-981a-744562df57d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:40.807852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee8b97-a255-4735-981a-744562df57d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:40.808145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee8b97-a255-4735-981a-744562df57d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:40.808463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee8b97-a255-4735-981a-744562df57d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:40.818627 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee8b97-a255-4735-981a-744562df57d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:40.820121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee8b97-a255-4735-981a-744562df57d5 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:40.831359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1026/2077] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:40.856671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:41.031087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd32417b-3d42-4a06-81db-a44df945e190 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:41.038179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:41.039271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:41.039608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:41.039846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] PATH_INFO: `/v3/users/47238795697740a290a2fb24e14e4282` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:41.042604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47238795697740a290a2fb24e14e4282)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:41.044867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:41.099855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:41.112155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47238795697740a290a2fb24e14e4282'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:41.123459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47238795697740a290a2fb24e14e4282'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:41.132508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47238795697740a290a2fb24e14e4282'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:41.133177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47238795697740a290a2fb24e14e4282'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:41.133912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47238795697740a290a2fb24e14e4282'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:41.155500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1052/2078] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:40 2022] DELETE /identity/v3/users/47238795697740a290a2fb24e14e4282 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:41.167071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:41.167071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:41.167071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:41.189124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:41.190333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:41.250413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1027/2079] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:41.262857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:41.264530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:41.333990 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3be674-4274-4e50-975f-cebf84286e67 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:41.337754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62772825-24eb-4e6a-b8dc-791b45896438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:41.338474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62772825-24eb-4e6a-b8dc-791b45896438 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:41.338780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62772825-24eb-4e6a-b8dc-791b45896438 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:41.339118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62772825-24eb-4e6a-b8dc-791b45896438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:41.349200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62772825-24eb-4e6a-b8dc-791b45896438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:41.350676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62772825-24eb-4e6a-b8dc-791b45896438 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:41.360873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1053/2080] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:42.349828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.351391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:42.364776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1ce5f-9b37-415d-895e-194d80385e2d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.369727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.370726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.371085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.371475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] PATH_INFO: `/v3/projects/e7e9685d896948a3a44c70a91459306d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.372820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7e9685d896948a3a44c70a91459306d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.374928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.426572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7e9685d896948a3a44c70a91459306d', 'request_id': 'req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:42.435861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7e9685d896948a3a44c70a91459306d', 'request_id': 'req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:42.443132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7e9685d896948a3a44c70a91459306d', 'request_id': 'req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:42.451120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:42.455285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] The token cache is being invalidate because project e7e9685d896948a3a44c70a91459306d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:42.455588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:42.456765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f3daa5f3-f4cf-4665-93b0-ca1a02fb89e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:42.459238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1028/2081] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:42 2022] DELETE /identity/v3/projects/e7e9685d896948a3a44c70a91459306d => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:42.464296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.464828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.465068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.485406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:42.486643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:42.554375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1054/2082] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:42.563971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.564347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.564738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.585620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:42.586822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:42.595598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1029/2083] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:42.600013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.610319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3a49d6-b2d0-48f1-84d9-7c0b3f29a532 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.614957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.615649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.615935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.616261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.616996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:42.617303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.618883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.637513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1055/2084] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:42.638659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.648654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7891e9d1-76cf-40e9-9833-7f88b84dadc6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.653376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.654060 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.654328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.654602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.655339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.657252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.682734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1030/2085] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:42 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:42.690744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.702696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c480f3-37b7-43be-a3e6-b2743014aef8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.750573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:42.771519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.772200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.772840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.772840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.773464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.775410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.797811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1056/2086] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:42 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:42.806571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.817364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e98ceb8-419d-4ef9-a42f-4d53ac699d0f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.822018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.822680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.822961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.823256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.823989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.825591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.837120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1031/2087] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:42.841110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.851091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dca027-8a01-449d-8479-3ee95759b320 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.856331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e269af6-fe18-4518-a20b-af33a4c24025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.857086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e269af6-fe18-4518-a20b-af33a4c24025 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.857434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e269af6-fe18-4518-a20b-af33a4c24025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.857777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e269af6-fe18-4518-a20b-af33a4c24025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.858570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e269af6-fe18-4518-a20b-af33a4c24025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.860262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e269af6-fe18-4518-a20b-af33a4c24025 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.870423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1057/2088] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:42.874599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:42.884648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92f9e7-aa0e-40d9-921b-aa8e517414ab admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:42.889462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:42.890338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.890621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.890895 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] PATH_INFO: `/v3/projects/6ec04c38f21e47128eca1d9e60617224/users/4953ac6f13524596ab00f2f18c2137d0/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.909899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ec04c38f21e47128eca1d9e60617224, user_id=4953ac6f13524596ab00f2f18c2137d0, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:42.913396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:42.926831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1032/2089] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:42 2022] PUT /identity/v3/projects/6ec04c38f21e47128eca1d9e60617224/users/4953ac6f13524596ab00f2f18c2137d0/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:42.937241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:42.937579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:42.937891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:42.982893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] MFA Rules not processed for user `4953ac6f13524596ab00f2f18c2137d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:42.982893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:43.043386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1058/2090] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:43.048395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.119604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddb9e60-1626-4fde-90f3-b50249c38664 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.126078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.127136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.127654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.128112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.129231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.132185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.187326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1033/2091] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:43 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:43.193149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.209515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c4186-9180-4b71-94cf-ab1b3d20e8ce None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.334247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.334964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.335279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.335567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.336274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.338396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.370865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1059/2092] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:43 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:43.374497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.387518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fd2272-c3f2-4804-9b20-dca337b58cbd admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.393662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.394345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.394634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.394974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.395939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.397705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.414143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1034/2093] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:43.418054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.430475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c177fc-6468-465e-a1b2-32f213dafecd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.438506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1961e8-8e73-445d-9ce0-1adf61a30a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.439525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1961e8-8e73-445d-9ce0-1adf61a30a8a admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.440011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1961e8-8e73-445d-9ce0-1adf61a30a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.440384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1961e8-8e73-445d-9ce0-1adf61a30a8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.441201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1961e8-8e73-445d-9ce0-1adf61a30a8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.443241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1961e8-8e73-445d-9ce0-1adf61a30a8a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.459262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1060/2094] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:43.461500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.474313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f963fa7-825a-41f2-af67-1ea73f0000d2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.489740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.491106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.491518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.491896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] PATH_INFO: `/v3/projects/12a325a673ad4215890c012512459507/users/69ebd38b58b243989655e0507be0b179/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.514470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a325a673ad4215890c012512459507, user_id=69ebd38b58b243989655e0507be0b179, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.517978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.540327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1035/2095] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:43 2022] PUT /identity/v3/projects/12a325a673ad4215890c012512459507/users/69ebd38b58b243989655e0507be0b179/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:43.551274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.552208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.552499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.583622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] MFA Rules not processed for user `69ebd38b58b243989655e0507be0b179`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:43.584857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:43.612538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:43.673674 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1061/2096] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:43.679022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.742030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:43.758878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19117f61-7c2c-4434-bccb-5230bbefd792 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.770360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.772735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.773256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.773779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.774504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.777012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.818309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1036/2097] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:43 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:43.822130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.836293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1146895d-f38c-4c63-a7e0-5801db61a130 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.919914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.920790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.921084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.921365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.922078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.924192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.946092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1062/2098] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:43 2022] POST /identity/v3/users => generated 299 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:43.950107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.961346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c93219-ad1a-430b-ae25-3f39799d7236 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:43.966127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:43.966797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:43.967084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:43.967401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:43.968160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:43.969780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:43.980556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1037/2099] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:43.985769 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:43.996080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baf0ee4-3d84-4c2f-b4df-f0ddeac38346 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:44.001676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:44.002581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:44.002868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:44.003150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] PATH_INFO: `/v3/projects/847dd3c0ba664441abd7f2448ffc7e61/users/93203c242ff24d89aa3fd5e36dea749c/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:44.021284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=847dd3c0ba664441abd7f2448ffc7e61, user_id=93203c242ff24d89aa3fd5e36dea749c, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:44.024097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:44.038592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1063/2100] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:43 2022] PUT /identity/v3/projects/847dd3c0ba664441abd7f2448ffc7e61/users/93203c242ff24d89aa3fd5e36dea749c/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:44.042898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:44.053857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34735b7d-c7b4-4688-a3a4-d4a196c483a4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:44.117076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce45f51-2730-4818-b0d9-102ed1eef95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:44.117753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce45f51-2730-4818-b0d9-102ed1eef95f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:44.118040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce45f51-2730-4818-b0d9-102ed1eef95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:44.118316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce45f51-2730-4818-b0d9-102ed1eef95f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:44.119071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce45f51-2730-4818-b0d9-102ed1eef95f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:44.120734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce45f51-2730-4818-b0d9-102ed1eef95f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:44.131422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1038/2101] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:44.135583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:44.154654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34850b-13fb-4960-bef8-49d6707fc6ad admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:44.169114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:44.170024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:44.170304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:44.170577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] PATH_INFO: `/v3/projects/847dd3c0ba664441abd7f2448ffc7e61/users/93203c242ff24d89aa3fd5e36dea749c/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:44.172590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=847dd3c0ba664441abd7f2448ffc7e61, user_id=93203c242ff24d89aa3fd5e36dea749c, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:44.175336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:44.189194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1064/2102] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:44 2022] PUT /identity/v3/projects/847dd3c0ba664441abd7f2448ffc7e61/users/93203c242ff24d89aa3fd5e36dea749c/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:44.200902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:44.201195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:44.201479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:44.228776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] MFA Rules not processed for user `93203c242ff24d89aa3fd5e36dea749c`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:44.230067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:44.304046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1039/2103] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:44.315058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:44.316849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:44.376756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a06c749-8914-4856-a9a2-acb3c53f0d34 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:44.380628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef1a85-759c-41b2-a559-bbee8396a5ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:44.381293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef1a85-759c-41b2-a559-bbee8396a5ac service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:44.381580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef1a85-759c-41b2-a559-bbee8396a5ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:44.381860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef1a85-759c-41b2-a559-bbee8396a5ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:44.412369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:44.446797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef1a85-759c-41b2-a559-bbee8396a5ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:44.448299 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef1a85-759c-41b2-a559-bbee8396a5ac service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:44.459806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1065/2104] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:50.056634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:50.058517 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.110215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:50.139174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b889d2c-bea4-45e3-ac63-0629c439beb8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:50.144548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:50.147765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.148054 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.148338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] PATH_INFO: `/v3/users/192a573e15a841b4b40e682f1754623e` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.150150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=192a573e15a841b4b40e682f1754623e)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:50.152382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:50.219717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192a573e15a841b4b40e682f1754623e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.229222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192a573e15a841b4b40e682f1754623e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.249269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192a573e15a841b4b40e682f1754623e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.260616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192a573e15a841b4b40e682f1754623e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.261339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192a573e15a841b4b40e682f1754623e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.268119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1040/2105] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:50 2022] DELETE /identity/v3/users/192a573e15a841b4b40e682f1754623e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:50.273104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.273603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.273725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.293010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:50.294423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.359251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1066/2106] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:50.369837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:50.371721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.447109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98c7cee-736c-4c66-b8f1-d27f768de6ed admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:50.448894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:50.449554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.452209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.452209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.461557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:50.463056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:50.474933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1041/2107] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:50.496054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:50.497607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.580401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbdedb-c757-4bf9-8b72-d3950827abbb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:50.586443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:50.587115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.587469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.588099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.590534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.679907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:50.679907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:50.699869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1067/2108] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:50.847127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:50.850120 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:50.894642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32810d88-1352-43ee-a170-8625edf739d3 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:50.899726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:50.900625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:50.900916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:50.901181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] PATH_INFO: `/v3/projects/3487ca3da7d1412fbaf0f999310ffda1` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:50.902467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3487ca3da7d1412fbaf0f999310ffda1)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:50.904418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:50.985140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3487ca3da7d1412fbaf0f999310ffda1', 'request_id': 'req-5a2a8270-d979-4ec5-8cd5-9075da886f0a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.994852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3487ca3da7d1412fbaf0f999310ffda1', 'request_id': 'req-5a2a8270-d979-4ec5-8cd5-9075da886f0a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:50.999256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:51.006889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3487ca3da7d1412fbaf0f999310ffda1', 'request_id': 'req-5a2a8270-d979-4ec5-8cd5-9075da886f0a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.022250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] The token cache is being invalidate because project 3487ca3da7d1412fbaf0f999310ffda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:51.022675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.024023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.029114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1042/2109] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:50 2022] DELETE /identity/v3/projects/3487ca3da7d1412fbaf0f999310ffda1 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:51.030535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:51.049997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1fe27d-97a4-4767-b37b-91ae5a3528c7 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:51.074198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:51.154135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:51.155199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.155730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.157639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] PATH_INFO: `/v3/users/cc40dae94ab6479886765225f77aa8d6` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.162072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc40dae94ab6479886765225f77aa8d6)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:51.164273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:51.262952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc40dae94ab6479886765225f77aa8d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.274623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc40dae94ab6479886765225f77aa8d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.284953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc40dae94ab6479886765225f77aa8d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.287530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc40dae94ab6479886765225f77aa8d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.288332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc40dae94ab6479886765225f77aa8d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.316435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1068/2110] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:51 2022] DELETE /identity/v3/users/cc40dae94ab6479886765225f77aa8d6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:51.466077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:51.468052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:51.531721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2a8270-d979-4ec5-8cd5-9075da886f0a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:51.594847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:51.595591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.595909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.596199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.598571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:51.627818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:51.629271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:51.683108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:51.684919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01fe7a8-ede4-42ae-aed9-684af147d0a1 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:51.697110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1043/2111] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:51.741902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f085a88-cc44-4d30-a4bb-474364184d81 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:51.748595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:51.749696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.750012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.750354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] PATH_INFO: `/v3/projects/53a0a3b1d78140d59e8c6151e86761d3` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.751763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] RBAC: Authorizing `identity:delete_project(project_id=53a0a3b1d78140d59e8c6151e86761d3)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:51.754235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:51.816277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:51.821962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53a0a3b1d78140d59e8c6151e86761d3', 'request_id': 'req-3c953536-374e-4baa-9dda-cb599417890b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.829439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53a0a3b1d78140d59e8c6151e86761d3', 'request_id': 'req-3c953536-374e-4baa-9dda-cb599417890b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.836526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53a0a3b1d78140d59e8c6151e86761d3', 'request_id': 'req-3c953536-374e-4baa-9dda-cb599417890b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.851357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] The token cache is being invalidate because project 53a0a3b1d78140d59e8c6151e86761d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:51.851789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.852437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3c953536-374e-4baa-9dda-cb599417890b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:51.854369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1069/2112] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:51 2022] DELETE /identity/v3/projects/53a0a3b1d78140d59e8c6151e86761d3 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:51.860083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.860380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.860710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.884429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:51.885660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:51.938963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:51.967061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1044/2113] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:51.969582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:51.969872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:51.970155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:51.992323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:51.994306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.008779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1070/2114] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:52.013662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.024731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefb83f3-7fdb-4c00-bb98-84c7228cd79a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.034205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.035402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.036493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.036928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.038011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:52.038323 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.040005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.054962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1045/2115] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:52.059519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.070625 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ff8a4c-1b2a-41fc-8d4e-032dd5455422 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.078555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.079306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.079641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.079924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.080675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.082698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.118163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1071/2116] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:52 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:52.122543 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.137291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a266a3-5180-4d87-b38c-de6fdbf3ef49 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.234041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.234720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.235003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.235310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.236010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.238048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.264403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1046/2117] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:52 2022] POST /identity/v3/users => generated 288 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:52.268258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.287594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfca793-d8a2-4701-9da4-3d3846d29f07 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.294392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.295351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.295798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.296312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.297268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.299001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.313188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1072/2118] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:52.317140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.327672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed9a7a-1868-4bad-9aea-36fb84edf9bc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.332428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c82547-0a6b-43d7-9874-ed3c4a3e83ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.333123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c82547-0a6b-43d7-9874-ed3c4a3e83ee admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.333406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c82547-0a6b-43d7-9874-ed3c4a3e83ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.333681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c82547-0a6b-43d7-9874-ed3c4a3e83ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.334455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c82547-0a6b-43d7-9874-ed3c4a3e83ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.336101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c82547-0a6b-43d7-9874-ed3c4a3e83ee admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.347383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1047/2119] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:52.352223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.362372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805cd06a-526d-460a-8746-df275920b071 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.373128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.374080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.374299 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.374611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] PATH_INFO: `/v3/projects/cc3f5568a3c74cf3ba1f85a51f67d258/users/ed505ac9e3614789bc37b5b96a532fc5/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.390312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc3f5568a3c74cf3ba1f85a51f67d258, user_id=ed505ac9e3614789bc37b5b96a532fc5, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.393404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.409868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1073/2120] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:52 2022] PUT /identity/v3/projects/cc3f5568a3c74cf3ba1f85a51f67d258/users/ed505ac9e3614789bc37b5b96a532fc5/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:52.421238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.421553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.421809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.454058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] MFA Rules not processed for user `ed505ac9e3614789bc37b5b96a532fc5`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:52.455849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.513236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.515039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.544535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1048/2121] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:52.548869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.590726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b7a009-a172-4c1d-a5bc-2e574e04ceb8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.595508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.596506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.596788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.597083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] PATH_INFO: `/v3/users/af57181073784691ba5562c83b65beea` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.598334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af57181073784691ba5562c83b65beea)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.600278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.600479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:52.617051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcc843e-1ab8-4ca9-a44b-18ba7b0e3de5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.622028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.622686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.622980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.623351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.624050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.625973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.655254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1049/2122] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:52 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:52.659837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.667505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af57181073784691ba5562c83b65beea'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.678933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af57181073784691ba5562c83b65beea'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.689261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af57181073784691ba5562c83b65beea'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.689933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af57181073784691ba5562c83b65beea'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.690720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-528a071c-625d-4fd3-9d9e-e68c192a0555 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af57181073784691ba5562c83b65beea'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:52.697173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=83,keystone:DELETE=13,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:52.712049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1074/2123] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:52 2022] DELETE /identity/v3/users/af57181073784691ba5562c83b65beea => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:52.716434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.716706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.716985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.732853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a22f6-4586-47a0-b0df-ddc40e90d850 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.735027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:52.736264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.797698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.798371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.798656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.798988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.799719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.800582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1075/2124] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:52.802331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.810755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.812364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.827245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1050/2125] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:52 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:52.831438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.833153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:52.845143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b48db5b-6768-4b7f-94df-0ecca2490477 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.850722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.851416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.851788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.851956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.852683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.854466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.870330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1051/2126] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:52.872567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.887110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec81c74f-af5a-4e45-ba99-758cb74d170c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.893046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.894174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.894633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.895027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad7996-9776-4e39-a3d9-82499083084f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:52.895344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] PATH_INFO: `/v3/projects/9824cf3a4b8b4a9faf37c6cffa70b318/users/88440f0aa7fc41d2aabcdad74f64b48c/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.898951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:52.899649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:52.899938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:52.901115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:52.911282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.912724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.913993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9824cf3a4b8b4a9faf37c6cffa70b318, user_id=88440f0aa7fc41d2aabcdad74f64b48c, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:52.918796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:52.923062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1076/2127] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:52.946322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1052/2128] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:52 2022] PUT /identity/v3/projects/9824cf3a4b8b4a9faf37c6cffa70b318/users/88440f0aa7fc41d2aabcdad74f64b48c/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:52.950792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:52.961043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daccc69-d5a6-4ef4-ba40-e5da542abb7e service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:53.045237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584346d1-073c-4c12-85f9-763d2bc69079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:53.046366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584346d1-073c-4c12-85f9-763d2bc69079 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.046854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584346d1-073c-4c12-85f9-763d2bc69079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.047472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584346d1-073c-4c12-85f9-763d2bc69079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.048475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584346d1-073c-4c12-85f9-763d2bc69079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:53.050395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584346d1-073c-4c12-85f9-763d2bc69079 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:53.063260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1077/2129] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:52 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:53.065976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:53.081214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fb1837-971e-405b-b397-d3f4ea809feb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:53.086906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:53.088003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.088289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.088567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] PATH_INFO: `/v3/projects/9824cf3a4b8b4a9faf37c6cffa70b318/users/88440f0aa7fc41d2aabcdad74f64b48c/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.090620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9824cf3a4b8b4a9faf37c6cffa70b318, user_id=88440f0aa7fc41d2aabcdad74f64b48c, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:53.094017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:53.106027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1053/2130] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:53 2022] PUT /identity/v3/projects/9824cf3a4b8b4a9faf37c6cffa70b318/users/88440f0aa7fc41d2aabcdad74f64b48c/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:53.116742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.117153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.117561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.200710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] MFA Rules not processed for user `88440f0aa7fc41d2aabcdad74f64b48c`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:53.201951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:53.219495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:53.323546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1078/2131] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:53 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:53.343194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:53.345149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:53.386264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:53.428719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6df63e-95b9-42da-a47a-573148543d62 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:53.434804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:53.435670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.436125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.436552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.446272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:53.516348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:53.518429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:53.532298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1054/2132] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:53.533247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:53.534660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:53.624752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de26730-7101-4681-8df7-f9ef91fbd672 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:53.630362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:53.631357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.631756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.632042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] PATH_INFO: `/v3/projects/095b1d6ba4e448a7ac328185abbb784f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.633270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=095b1d6ba4e448a7ac328185abbb784f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:53.635375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:53.685015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '095b1d6ba4e448a7ac328185abbb784f', 'request_id': 'req-1658390e-b279-44f7-993e-e8f598e62621'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.692955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '095b1d6ba4e448a7ac328185abbb784f', 'request_id': 'req-1658390e-b279-44f7-993e-e8f598e62621'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.700076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '095b1d6ba4e448a7ac328185abbb784f', 'request_id': 'req-1658390e-b279-44f7-993e-e8f598e62621'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.714348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] The token cache is being invalidate because project 095b1d6ba4e448a7ac328185abbb784f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:53.714743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.715419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.717123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1079/2133] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:53 2022] DELETE /identity/v3/projects/095b1d6ba4e448a7ac328185abbb784f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:53.721551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:53.734166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bfcab7-75da-461d-8590-b48270e57b83 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:53.795588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:53.796465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:53.796748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:53.797191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] PATH_INFO: `/v3/users/763b856a0dd24ad584bf322e6fc41f80` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:53.798395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=763b856a0dd24ad584bf322e6fc41f80)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:53.800381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:53.858881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '763b856a0dd24ad584bf322e6fc41f80'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.869637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '763b856a0dd24ad584bf322e6fc41f80'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.883542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '763b856a0dd24ad584bf322e6fc41f80'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.888847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:53.891081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '763b856a0dd24ad584bf322e6fc41f80'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.892170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b7daed04-a733-4af5-a0d3-cf74eb51ccc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '763b856a0dd24ad584bf322e6fc41f80'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:53.900870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1055/2134] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:53 2022] DELETE /identity/v3/users/763b856a0dd24ad584bf322e6fc41f80 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:54.177106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.179203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.192720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1658390e-b279-44f7-993e-e8f598e62621 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.236804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:54.255303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.256192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.256567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.256803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] PATH_INFO: `/v3/projects/f40f2a26410a4a69a9a941505a07694f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.258007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f40f2a26410a4a69a9a941505a07694f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.260099 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.315436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f40f2a26410a4a69a9a941505a07694f', 'request_id': 'req-43fffe42-efb3-4014-80eb-87128bc27b95'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.324093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f40f2a26410a4a69a9a941505a07694f', 'request_id': 'req-43fffe42-efb3-4014-80eb-87128bc27b95'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.331666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f40f2a26410a4a69a9a941505a07694f', 'request_id': 'req-43fffe42-efb3-4014-80eb-87128bc27b95'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.351919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] The token cache is being invalidate because project f40f2a26410a4a69a9a941505a07694f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:54.352154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.352835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-43fffe42-efb3-4014-80eb-87128bc27b95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:54.354549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1080/2135] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:54 2022] DELETE /identity/v3/projects/f40f2a26410a4a69a9a941505a07694f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:54.359909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.360584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.360941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.381368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:54.382809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.462806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1056/2136] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:54.470733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.471010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.471349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.490414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:54.491841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.503899 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1081/2137] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:54.508449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.520652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792504b-f29e-4e81-87b9-60b9c409083e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.526737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.527587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.527948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.528229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.529418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:54.529732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.531418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.543988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1057/2138] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:54.549225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.560283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acc0c33-27db-4850-99a4-a338b9cfd8bc None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.566247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.567726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.568083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.568442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.569290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.571554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.602390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1082/2139] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:54.612872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.623977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c326d-1063-4d4f-8d11-c9e2a22d1385 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.685137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.685819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.686101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.686383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.687121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.689107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.719279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1058/2140] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:54.723712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.734786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a38d52-5db0-41fc-be41-caeeed15c294 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.740196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.740871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.741206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.741431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.742160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.743793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.758118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1083/2141] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:54.759406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.769706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d9ed53-0d27-4d7d-8cb7-73c4f9e5b784 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.775467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6778e500-1b17-4dba-8956-34f31d97263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.776141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e500-1b17-4dba-8956-34f31d97263a admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.776428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e500-1b17-4dba-8956-34f31d97263a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.776734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6778e500-1b17-4dba-8956-34f31d97263a admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.777508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e500-1b17-4dba-8956-34f31d97263a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.779116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6778e500-1b17-4dba-8956-34f31d97263a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.789456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1059/2142] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:54.793891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:54.803882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f04da5-d675-4138-a623-d776a8c4c733 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:54.808345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:54.809214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.809492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.809767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] PATH_INFO: `/v3/projects/8333398c61d74e498fed189ddc4484e8/users/7b6a54f8d1404754ae3708bcab774072/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.825046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8333398c61d74e498fed189ddc4484e8, user_id=7b6a54f8d1404754ae3708bcab774072, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:54.827724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:54.840983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1084/2143] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:54 2022] PUT /identity/v3/projects/8333398c61d74e498fed189ddc4484e8/users/7b6a54f8d1404754ae3708bcab774072/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:54.851847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:54.852215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:54.852569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:54.879943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] MFA Rules not processed for user `7b6a54f8d1404754ae3708bcab774072`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:54.881222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:54.933059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:54.952993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1060/2144] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:54.957690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.021467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fddb1f-61c7-4632-aa8e-fe4abb64aa23 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.026098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.026807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.027092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.027404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.028108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.030003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.042860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:55.061710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1085/2145] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:54 2022] POST /identity/v3/projects => generated 363 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:55.062866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.074004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58ebcc-6e1b-44fe-8498-6c11aa933f49 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.133430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.134093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.134372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.134646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.135366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.137288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.156854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1061/2146] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:55.160891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.171921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc525b-0cfd-4696-952f-bedc096cf062 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.183417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.184119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.184400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.184673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.185403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.187085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.198959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1086/2147] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:55.203108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.213316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b490ef5-d7c8-47e8-923c-4e63b6bf07ee admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.218071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.218977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.219293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.219578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] PATH_INFO: `/v3/projects/df6cd5432d1440fc94cc210c5b74e22a/users/b83f2c9505374877b5511f4d60ddcd5a/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.234673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6cd5432d1440fc94cc210c5b74e22a, user_id=b83f2c9505374877b5511f4d60ddcd5a, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.237387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.250321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1062/2148] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:55 2022] PUT /identity/v3/projects/df6cd5432d1440fc94cc210c5b74e22a/users/b83f2c9505374877b5511f4d60ddcd5a/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:55.254337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.264675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804fa3f6-52e7-4e5f-b9b7-359a00bb6ca4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.324916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134ecc9b-de38-48e8-949c-dde26643f96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.325718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ecc9b-de38-48e8-949c-dde26643f96e admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.326102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ecc9b-de38-48e8-949c-dde26643f96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.326482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134ecc9b-de38-48e8-949c-dde26643f96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.327390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ecc9b-de38-48e8-949c-dde26643f96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.329205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134ecc9b-de38-48e8-949c-dde26643f96e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.339731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1087/2149] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:55.344039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.354610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27252e7-43fd-415b-80c2-db1ddb4e1582 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.359223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.360118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.360401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.360678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] PATH_INFO: `/v3/projects/df6cd5432d1440fc94cc210c5b74e22a/users/b83f2c9505374877b5511f4d60ddcd5a/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.362624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6cd5432d1440fc94cc210c5b74e22a, user_id=b83f2c9505374877b5511f4d60ddcd5a, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.376130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.389619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1063/2150] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:55 2022] PUT /identity/v3/projects/df6cd5432d1440fc94cc210c5b74e22a/users/b83f2c9505374877b5511f4d60ddcd5a/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:55.399946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.400238 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.400513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.428699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] MFA Rules not processed for user `b83f2c9505374877b5511f4d60ddcd5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:55.429924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.499390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1088/2151] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:55 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:55.513315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.515077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.590823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667c38c-c301-4890-8cac-a88facc74ee3 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.603623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.607502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.607799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.608086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.622258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.622633 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.635145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1064/2152] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:55.755284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.758433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.771437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:55.773026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:55.774680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ee00d-ccc9-4ef2-9ca2-4d59ca4f3846 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.778386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4476d4a-0a59-4c1b-bcd6-a6fe7b1f821f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.779402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4476d4a-0a59-4c1b-bcd6-a6fe7b1f821f service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.779813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4476d4a-0a59-4c1b-bcd6-a6fe7b1f821f service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.780200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4476d4a-0a59-4c1b-bcd6-a6fe7b1f821f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.832779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:55.855907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1fe137-586e-42fc-818d-2ccda0fa60f6 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:55.862629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:55.865397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4476d4a-0a59-4c1b-bcd6-a6fe7b1f821f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.866317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:55.866523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:55.866906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] PATH_INFO: `/v3/users/4953ac6f13524596ab00f2f18c2137d0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:55.867024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4476d4a-0a59-4c1b-bcd6-a6fe7b1f821f service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.868512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4953ac6f13524596ab00f2f18c2137d0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:55.870714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:55.879925 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1089/2153] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:55 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:55.965786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4953ac6f13524596ab00f2f18c2137d0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:55.974728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4953ac6f13524596ab00f2f18c2137d0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:55.996385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4953ac6f13524596ab00f2f18c2137d0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.011814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4953ac6f13524596ab00f2f18c2137d0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.012752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4953ac6f13524596ab00f2f18c2137d0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.031191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1065/2154] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:55 2022] DELETE /identity/v3/users/4953ac6f13524596ab00f2f18c2137d0 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:56.036085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.036367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.036647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.042693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:56.062165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:56.063780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:56.179919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1090/2155] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:56.190130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.191764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:56.259366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c5973-9f93-4e34-99ac-02d07fbc5a2b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.263364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.263976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.264258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.264534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.274282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.275770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.286302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1066/2156] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:56.566500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.568275 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:56.581072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918c7c6-bcd1-4ba4-99b6-0592f1096584 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.586222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.587087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.587398 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.587701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] PATH_INFO: `/v3/projects/6ec04c38f21e47128eca1d9e60617224` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.588880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ec04c38f21e47128eca1d9e60617224)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.590777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.637671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ec04c38f21e47128eca1d9e60617224', 'request_id': 'req-5657ca41-69b1-40ca-b784-96c8f2f93039'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.643270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ec04c38f21e47128eca1d9e60617224', 'request_id': 'req-5657ca41-69b1-40ca-b784-96c8f2f93039'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.650120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ec04c38f21e47128eca1d9e60617224', 'request_id': 'req-5657ca41-69b1-40ca-b784-96c8f2f93039'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.662805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] The token cache is being invalidate because project 6ec04c38f21e47128eca1d9e60617224 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:56.663093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.663763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.665345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1091/2157] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:56 2022] DELETE /identity/v3/projects/6ec04c38f21e47128eca1d9e60617224 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:56.670015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.680276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753b76d-1b9d-4474-9d36-ea99220a5aa9 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.700112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:56.701603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:56.782414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.783347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.783640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.783922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] PATH_INFO: `/v3/users/69ebd38b58b243989655e0507be0b179` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.785243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69ebd38b58b243989655e0507be0b179)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.787353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.804687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657ca41-69b1-40ca-b784-96c8f2f93039 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:56.809321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:56.809466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:56.810449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.810592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.810883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] PATH_INFO: `/v3/users/b2e868e843fa42f89c9bfe49459531a9` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.812143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e868e843fa42f89c9bfe49459531a9)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:56.814932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:56.847908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:56.866629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69ebd38b58b243989655e0507be0b179'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.871091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e868e843fa42f89c9bfe49459531a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.874554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69ebd38b58b243989655e0507be0b179'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.879806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e868e843fa42f89c9bfe49459531a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.887578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e868e843fa42f89c9bfe49459531a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.887854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69ebd38b58b243989655e0507be0b179'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.888218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e868e843fa42f89c9bfe49459531a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.888875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e868e843fa42f89c9bfe49459531a9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.895879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69ebd38b58b243989655e0507be0b179'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.896534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6aff50ef-a16e-4284-a7bd-95dced091fc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69ebd38b58b243989655e0507be0b179'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:56.903064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1067/2158] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:56 2022] DELETE /identity/v3/users/69ebd38b58b243989655e0507be0b179 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:56.910820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1092/2159] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:56 2022] DELETE /identity/v3/users/b2e868e843fa42f89c9bfe49459531a9 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:56.915165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:56.915494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:56.915771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:56.932856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:56.934047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:56.997140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1068/2160] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:57.013423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.015707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.086078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4348782d-c399-42c4-8282-3d65d6c33272 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.090476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.091184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.091621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.091889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.104008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.105630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.116833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1093/2161] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:57.246911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.249386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.272994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05b7570-3777-49aa-9da9-de86f78928fe None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.280753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.280753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.280753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.280753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] PATH_INFO: `/v3/users/ed505ac9e3614789bc37b5b96a532fc5` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.282025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.282374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed505ac9e3614789bc37b5b96a532fc5)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.283558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.284708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.306156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a52d1-337b-418f-8ba5-966a8d406c9a service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.311248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.311906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.312193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.312475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] PATH_INFO: `/v3/projects/12a325a673ad4215890c012512459507` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.313742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a325a673ad4215890c012512459507)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.317048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.378827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed505ac9e3614789bc37b5b96a532fc5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.399624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed505ac9e3614789bc37b5b96a532fc5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.432242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a325a673ad4215890c012512459507', 'request_id': 'req-3909f310-0631-4a41-ac4b-443c0ce03eec'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.434902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed505ac9e3614789bc37b5b96a532fc5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.440645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a325a673ad4215890c012512459507', 'request_id': 'req-3909f310-0631-4a41-ac4b-443c0ce03eec'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.449246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a325a673ad4215890c012512459507', 'request_id': 'req-3909f310-0631-4a41-ac4b-443c0ce03eec'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.460998 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed505ac9e3614789bc37b5b96a532fc5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.461952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f17b7a98-c0c3-4cca-87da-590e0df4d9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed505ac9e3614789bc37b5b96a532fc5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.465083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:57.474449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] The token cache is being invalidate because project 12a325a673ad4215890c012512459507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:57.475136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.475743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.477854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1094/2162] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:57 2022] DELETE /identity/v3/projects/12a325a673ad4215890c012512459507 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:57.480241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1069/2163] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:57 2022] DELETE /identity/v3/users/ed505ac9e3614789bc37b5b96a532fc5 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:57.484730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.486686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.486978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.488255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.489207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.530989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:57.532577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.543510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:57.588514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3909f310-0631-4a41-ac4b-443c0ce03eec admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.637406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1070/2164] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:57.638331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.645511 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.660057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c088e26-5839-42fc-a868-00af3bd3ac6b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.669032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.670106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.670452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.670782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] PATH_INFO: `/v3/projects/8b4f777ff73849638c19d79a4e38cb49` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.672538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b4f777ff73849638c19d79a4e38cb49)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.674520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.675575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.675853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.676730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] PATH_INFO: `/v3/users/93203c242ff24d89aa3fd5e36dea749c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.676869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.678029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93203c242ff24d89aa3fd5e36dea749c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.680502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.737277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b4f777ff73849638c19d79a4e38cb49', 'request_id': 'req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.749421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93203c242ff24d89aa3fd5e36dea749c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.749667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b4f777ff73849638c19d79a4e38cb49', 'request_id': 'req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.756568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b4f777ff73849638c19d79a4e38cb49', 'request_id': 'req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.758067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93203c242ff24d89aa3fd5e36dea749c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.766475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93203c242ff24d89aa3fd5e36dea749c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.767281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93203c242ff24d89aa3fd5e36dea749c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.767908 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-78ff03a1-fe08-4138-bd73-68c259e44184 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93203c242ff24d89aa3fd5e36dea749c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.768430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] The token cache is being invalidate because project 8b4f777ff73849638c19d79a4e38cb49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:57.768712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.769381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:57.771594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1071/2165] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:57 2022] DELETE /identity/v3/projects/8b4f777ff73849638c19d79a4e38cb49 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:57.772852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.774146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.787164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1095/2166] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:57 2022] DELETE /identity/v3/users/93203c242ff24d89aa3fd5e36dea749c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:57.789575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5666c-7e54-422a-8d3b-02074cd2859c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.789957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5666c-7e54-422a-8d3b-02074cd2859c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.790254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5666c-7e54-422a-8d3b-02074cd2859c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.812778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-53c5666c-7e54-422a-8d3b-02074cd2859c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:57.814070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-53c5666c-7e54-422a-8d3b-02074cd2859c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.886707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b29dd8-f7c4-422e-aa10-cfe32a2b9cfd admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.896205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:57.912630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1096/2167] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:57.919966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.920381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.920844 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.949613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:57.950842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.955756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:57.963276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1097/2168] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:57.964471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:57.973941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.974691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.974847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd307ba-5089-45e6-9f3d-0ebb7fa55996 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:57.975401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.975796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.977958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:57.979927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:57.980688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:57.981075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:57.981426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:57.982216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:57.982599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.984335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:57.993984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:57.997515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.009797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1072/2169] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:58.013736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1098/2170] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:58.019312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.038688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8584e4-b224-48fa-bc48-1a77ccc920f1 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.044829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.045493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.045772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.046048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.046742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.048898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.092422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1073/2171] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/projects => generated 375 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:58.093603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.106150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7439dbc4-8994-4c87-aadc-f41f50d29e94 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.172512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.173447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.174284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.174284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.175333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.177936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.206173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1099/2172] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:42:58.207515 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.222105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020ff0cf-701d-4157-b479-5712b391e587 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.231123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.231906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.232262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.232550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.233305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.235348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.252492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1074/2173] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:42:58.253771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.264395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.266678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b382786-635e-4e17-8de7-b717bd822e01 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.268858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.272946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.273635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.274112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.274783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.275855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.277275 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.294629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba86ffa0-d814-469d-bf99-a18001831b6e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.299590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1100/2174] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:58.300161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.301515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.301636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.301636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.302044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] PATH_INFO: `/v3/projects/847dd3c0ba664441abd7f2448ffc7e61` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.303131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=847dd3c0ba664441abd7f2448ffc7e61)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.305107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.313528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473575d3-39d7-4e76-bd4f-44596e65826b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.320277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5401e54a-ffbd-4c42-89d8-6458603332e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.321271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401e54a-ffbd-4c42-89d8-6458603332e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.321561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401e54a-ffbd-4c42-89d8-6458603332e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.321854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5401e54a-ffbd-4c42-89d8-6458603332e4 admin admin] PATH_INFO: `/v3/projects/46c24775038b45598100e637f569a117/users/19cee9d773b541e186dfe7b90aa73e05/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.357518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401e54a-ffbd-4c42-89d8-6458603332e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46c24775038b45598100e637f569a117, user_id=19cee9d773b541e186dfe7b90aa73e05, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.364228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5401e54a-ffbd-4c42-89d8-6458603332e4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.382866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1101/2175] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:58 2022] PUT /identity/v3/projects/46c24775038b45598100e637f569a117/users/19cee9d773b541e186dfe7b90aa73e05/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:58.391046 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '847dd3c0ba664441abd7f2448ffc7e61', 'request_id': 'req-65838d31-ff85-4231-a2b1-91e464553b53'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.394961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.395538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.396424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.400896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '847dd3c0ba664441abd7f2448ffc7e61', 'request_id': 'req-65838d31-ff85-4231-a2b1-91e464553b53'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.409074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '847dd3c0ba664441abd7f2448ffc7e61', 'request_id': 'req-65838d31-ff85-4231-a2b1-91e464553b53'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.421976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] The token cache is being invalidate because project 847dd3c0ba664441abd7f2448ffc7e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:58.422275 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.422978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-65838d31-ff85-4231-a2b1-91e464553b53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.424818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1075/2176] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:58 2022] DELETE /identity/v3/projects/847dd3c0ba664441abd7f2448ffc7e61 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:58.429979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.430271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.430548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.430667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] MFA Rules not processed for user `19cee9d773b541e186dfe7b90aa73e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:58.431961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.452415 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:58.453856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.513461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1102/2177] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:58.514374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.515723 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.543294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1076/2178] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:58.543294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.543563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.559606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:58.579084 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:58.604153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff7ea91-3d0c-48d0-b568-91e356778e21 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.604789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcbc862-326c-46b1-8483-84c811a6cfd5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.608184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3af788-3f34-4d28-b02e-85887f45859a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.608831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3af788-3f34-4d28-b02e-85887f45859a service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.608938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.609136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3af788-3f34-4d28-b02e-85887f45859a service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.609416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3af788-3f34-4d28-b02e-85887f45859a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.609759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.610033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.610379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] PATH_INFO: `/v3/projects/cc3f5568a3c74cf3ba1f85a51f67d258` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.611572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc3f5568a3c74cf3ba1f85a51f67d258)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.614020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.618454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3af788-3f34-4d28-b02e-85887f45859a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.619917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3af788-3f34-4d28-b02e-85887f45859a service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.628801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1077/2179] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:42:58.630707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.630995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.631303 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.650591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:58.651890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.667738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc3f5568a3c74cf3ba1f85a51f67d258', 'request_id': 'req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.675471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc3f5568a3c74cf3ba1f85a51f67d258', 'request_id': 'req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.682946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc3f5568a3c74cf3ba1f85a51f67d258', 'request_id': 'req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.699936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] The token cache is being invalidate because project cc3f5568a3c74cf3ba1f85a51f67d258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:58.700226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.701294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.702781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1103/2180] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:58 2022] DELETE /identity/v3/projects/cc3f5568a3c74cf3ba1f85a51f67d258 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:58.706850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.708370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:58.718172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1078/2181] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:58.722261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.722860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec33c4-7e46-48cd-b1a5-54d1fbdcd4d8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.727752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.728635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.728922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.729282 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] PATH_INFO: `/v3/users/88440f0aa7fc41d2aabcdad74f64b48c` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.730792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88440f0aa7fc41d2aabcdad74f64b48c)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.731656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd1d75b-3dd7-4beb-9776-2845dafc08cd None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.733462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.736974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.737654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.737943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.738222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.739518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:58.739826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.741431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.758211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1079/2182] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:42:58.759254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.770243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9900b7c-3a16-44cf-81dc-107358ce3ca7 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:58.777820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:58.778685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:58.778974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:58.779294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:58.780174 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:58.782920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:58.800693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88440f0aa7fc41d2aabcdad74f64b48c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.819196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88440f0aa7fc41d2aabcdad74f64b48c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.823912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1080/2183] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:58.825791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:58.836168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88440f0aa7fc41d2aabcdad74f64b48c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.837834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88440f0aa7fc41d2aabcdad74f64b48c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.839933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88440f0aa7fc41d2aabcdad74f64b48c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:58.892337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1104/2184] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:42:58 2022] DELETE /identity/v3/users/88440f0aa7fc41d2aabcdad74f64b48c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:42:59.016823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81795de0-562b-40d0-bbd5-525168ec8aa4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.171698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:59.203143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.205670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.205967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.206753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.207499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.209636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.240146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1081/2185] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:58 2022] POST /identity/v3/users => generated 293 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:59.243869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.245320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.265534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304d6ff5-3ea2-4b8e-9b56-1bb3e79006e0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.271044 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5185e1-1e9c-4fb8-a912-b1f4ed6f623f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.271868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5185e1-1e9c-4fb8-a912-b1f4ed6f623f admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.272248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5185e1-1e9c-4fb8-a912-b1f4ed6f623f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.272548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5185e1-1e9c-4fb8-a912-b1f4ed6f623f admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.273390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5185e1-1e9c-4fb8-a912-b1f4ed6f623f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.289302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5185e1-1e9c-4fb8-a912-b1f4ed6f623f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.309819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1105/2186] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:59.312896 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.327795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269d2dd-38d5-4ba5-bab6-d143be42b437 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.334509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.335642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.336147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.336638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.337583 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.351634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.354371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87495f-2231-4ced-afa6-3a19896220bb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.354815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87495f-2231-4ced-afa6-3a19896220bb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.355242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee87495f-2231-4ced-afa6-3a19896220bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.368655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1082/2187] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:42:59.370177 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.382572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-ee87495f-2231-4ced-afa6-3a19896220bb None None] MFA Rules not processed for user `46097d36b6fc4df5b510d279a4648eb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:59.384305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ee87495f-2231-4ced-afa6-3a19896220bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.389154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cc35a3-e3b4-4f7e-b4a7-aeb96f8a8564 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.397242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.400073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:59.411674 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.412008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.412341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] PATH_INFO: `/v3/projects/a862170421a346088ab15bc89c9a5716/users/4dbbf627f3bf4ffb87c6e23fac0b5002/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.448014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a862170421a346088ab15bc89c9a5716, user_id=4dbbf627f3bf4ffb87c6e23fac0b5002, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.451871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.472534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1083/2188] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:42:59 2022] PUT /identity/v3/projects/a862170421a346088ab15bc89c9a5716/users/4dbbf627f3bf4ffb87c6e23fac0b5002/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:59.475417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.475417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.494789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1106/2189] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:59.501390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.501578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.501891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.527048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:59.530373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.586969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03d508a-d22b-41d0-909f-995a6d8a4b99 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.601685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.606158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.606531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.607050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] PATH_INFO: `/v3/projects/9824cf3a4b8b4a9faf37c6cffa70b318` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.608811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9824cf3a4b8b4a9faf37c6cffa70b318)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.611507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.627009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1107/2190] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:59.636855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.648357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.673393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9824cf3a4b8b4a9faf37c6cffa70b318', 'request_id': 'req-1b937588-bc31-4df3-9e3c-28687c683e19'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:59.684641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9824cf3a4b8b4a9faf37c6cffa70b318', 'request_id': 'req-1b937588-bc31-4df3-9e3c-28687c683e19'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:59.694822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9824cf3a4b8b4a9faf37c6cffa70b318', 'request_id': 'req-1b937588-bc31-4df3-9e3c-28687c683e19'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:59.707074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] The token cache is being invalidate because project 9824cf3a4b8b4a9faf37c6cffa70b318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:42:59.707501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:59.708270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1b937588-bc31-4df3-9e3c-28687c683e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:42:59.711803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1084/2191] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:42:59 2022] DELETE /identity/v3/projects/9824cf3a4b8b4a9faf37c6cffa70b318 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:42:59.716473 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e3e24-1230-4c21-aae5-2dc8e91c4f2c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.716849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e3e24-1230-4c21-aae5-2dc8e91c4f2c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.717063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e3e24-1230-4c21-aae5-2dc8e91c4f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.732981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9423eb1a-3877-4124-a7f2-beed6b8d4d38 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.743988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-505e3e24-1230-4c21-aae5-2dc8e91c4f2c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:59.745210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-505e3e24-1230-4c21-aae5-2dc8e91c4f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.780463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:59.792694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.793567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.793866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.794215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.796270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.815538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1085/2192] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:42:59.823280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.823765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.824257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.829846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:42:59.844865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:42:59.846440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:42:59.855834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1086/2193] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:42:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:42:59.860214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.871049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef5a60c-c0d1-488a-bf14-bbc6abfc4f10 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.874017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.876030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.876154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.876730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.877026 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.877332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.878068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:42:59.878396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.880181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.887793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1108/2194] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:42:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:42:59.895262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1087/2195] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:42:59.897939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.909248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3138b7f7-6c48-4689-9889-0df21fc591e6 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:42:59.916956 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:42:59.917649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:42:59.918000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:42:59.918273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:42:59.918995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:42:59.921126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:42:59.958357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1109/2196] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:42:59 2022] POST /identity/v3/projects => generated 375 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:42:59.959642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:42:59.976525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb2afd-6ed1-4ac5-8f8a-3d1863fec67e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:00.050554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:00.051859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.051859 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.052134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.052784 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:00.054779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:00.082045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1088/2197] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:42:59 2022] POST /identity/v3/users => generated 288 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:00.087229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:00.119729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3e206-ac6f-48b5-b227-d28923a5a791 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:00.127518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:00.128206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.128589 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.128835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.129533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:00.134702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:00.152433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1110/2198] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:00.157299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:00.185547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbaf776-6c81-4dfa-8798-5190f9d37ffe admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:00.191670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9dc7c6-9338-4730-ae9c-4bd99405d7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:00.192459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dc7c6-9338-4730-ae9c-4bd99405d7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.192816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dc7c6-9338-4730-ae9c-4bd99405d7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.193155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9dc7c6-9338-4730-ae9c-4bd99405d7c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.194246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9dc7c6-9338-4730-ae9c-4bd99405d7c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:00.195946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9dc7c6-9338-4730-ae9c-4bd99405d7c1 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:00.232720 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:00.238277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1089/2199] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:00.243632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac016f3-c917-41ea-917d-af7fe0bcd1d3 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:00.250820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf391ebe-d254-4c92-9f5e-7ab73b561f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:00.252429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf391ebe-d254-4c92-9f5e-7ab73b561f78 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.253114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf391ebe-d254-4c92-9f5e-7ab73b561f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.253596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf391ebe-d254-4c92-9f5e-7ab73b561f78 admin admin] PATH_INFO: `/v3/projects/f72ee4751e0747d9af27ca355dde6c1e/users/d1c2aee42e2c40fe9c03106075289069/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.274605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf391ebe-d254-4c92-9f5e-7ab73b561f78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f72ee4751e0747d9af27ca355dde6c1e, user_id=d1c2aee42e2c40fe9c03106075289069, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:00.279834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf391ebe-d254-4c92-9f5e-7ab73b561f78 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:00.301093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.305197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.305623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.346349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1111/2200] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:00 2022] PUT /identity/v3/projects/f72ee4751e0747d9af27ca355dde6c1e/users/d1c2aee42e2c40fe9c03106075289069/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:00.357319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35fd6e-66a9-4ef4-80d9-8a5218d8ca1f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.357657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35fd6e-66a9-4ef4-80d9-8a5218d8ca1f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.357941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35fd6e-66a9-4ef4-80d9-8a5218d8ca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.359632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] MFA Rules not processed for user `19cee9d773b541e186dfe7b90aa73e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:00.361103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:00.425268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4c35fd6e-66a9-4ef4-80d9-8a5218d8ca1f None None] MFA Rules not processed for user `d1c2aee42e2c40fe9c03106075289069`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:00.428627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4c35fd6e-66a9-4ef4-80d9-8a5218d8ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:00.509568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1090/2201] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:00.519489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1112/2202] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:00.530805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:00.532289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:00.590062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:00.608219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1c5b2a-0174-456d-91f9-143ac977f9c1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:00.612591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:00.612691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:00.612980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:00.613232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:00.622868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:00.624377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:00.634011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1091/2203] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:05.625561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:05.626101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:05.626203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:05.652792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] MFA Rules not processed for user `4dbbf627f3bf4ffb87c6e23fac0b5002`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:05.653985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:05.698073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:05.718688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1113/2204] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:05.724282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:05.725844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:05.793619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91890d2c-f22d-4d91-ac51-126a6cda9042 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:05.798157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:05.798808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:05.799088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:05.799395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:05.800082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:05.802560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:05.826351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1092/2205] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:05 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:05.830460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:05.841493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf14576-0f5a-4ebe-95d6-31d010e03ff6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:05.900289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:05.901022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:05.901325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:05.901603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:05.902294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:05.904371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:05.930722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1114/2206] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:05 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:05.933023 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:05.948239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f3f1c-b049-4d81-bec3-bdc4d4119e98 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:05.958962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:05.959813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:05.960142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:05.960549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:05.961369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:05.963414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:05.977602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1093/2207] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:05.982071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:05.997789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e7a77-c036-42b8-a35f-f9b4f3babadc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:06.002662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:06.003608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:06.004122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:06.004455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] PATH_INFO: `/v3/projects/ca6b6983ee7342f993e12bb5cd0e0afd/users/3cb45eb7f35848ea86cffb221f64f1dd/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:06.023025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6b6983ee7342f993e12bb5cd0e0afd, user_id=3cb45eb7f35848ea86cffb221f64f1dd, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:06.025817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:06.041354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1115/2208] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:05 2022] PUT /identity/v3/projects/ca6b6983ee7342f993e12bb5cd0e0afd/users/3cb45eb7f35848ea86cffb221f64f1dd/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:06.045698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:06.057721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d639e2-1045-40ed-8fc5-238b2bbb255b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:06.158581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:06.159447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:06.160211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:06.161366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:06.163144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:06.168973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:06.189871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1094/2209] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:06 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:06.194337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:06.206378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ca3e8d-a661-4e84-b732-2ad171b043e3 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:06.220582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3c489-426a-436b-a735-35445d2038ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:06.221922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c489-426a-436b-a735-35445d2038ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:06.222227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c489-426a-436b-a735-35445d2038ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:06.223387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3c489-426a-436b-a735-35445d2038ae admin admin] PATH_INFO: `/v3/projects/ca6b6983ee7342f993e12bb5cd0e0afd/users/3cb45eb7f35848ea86cffb221f64f1dd/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:06.227479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c489-426a-436b-a735-35445d2038ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca6b6983ee7342f993e12bb5cd0e0afd, user_id=3cb45eb7f35848ea86cffb221f64f1dd, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:06.230466 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3c489-426a-436b-a735-35445d2038ae admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:06.243747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1116/2210] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:06 2022] PUT /identity/v3/projects/ca6b6983ee7342f993e12bb5cd0e0afd/users/3cb45eb7f35848ea86cffb221f64f1dd/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:08.474327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:08.500096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb86efb7-b362-4002-b7b8-8407d35c8136 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:08.639453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:08.670124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:08.672288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:08.672835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:08.674107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] PATH_INFO: `/v3/users/d1c2aee42e2c40fe9c03106075289069` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:08.676143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c2aee42e2c40fe9c03106075289069)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:08.678846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:08.767066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c2aee42e2c40fe9c03106075289069'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.783618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c2aee42e2c40fe9c03106075289069'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.818471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c2aee42e2c40fe9c03106075289069'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.828324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c2aee42e2c40fe9c03106075289069'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.829637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c2aee42e2c40fe9c03106075289069'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:08.840345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1095/2211] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:08 2022] DELETE /identity/v3/users/d1c2aee42e2c40fe9c03106075289069 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:08.845287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:08.845694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:08.845857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:08.898092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:08.900180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:09.102910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1117/2212] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:09.137365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:09.140995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:09.242204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718a60bd-9522-40b7-bba0-36951ede9138 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:09.247954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:09.248800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:09.255319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:09.256257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:09.272881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:09.274719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:09.288372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1096/2213] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:10.015202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.016795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.044265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b439a60e-f410-4b40-ae07-e66624a342cd None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.053881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.055043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.055563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.056010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] PATH_INFO: `/v3/users/7b6a54f8d1404754ae3708bcab774072` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.061489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6a54f8d1404754ae3708bcab774072)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.063745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.137650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.139346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.153474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:10.171940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6a54f8d1404754ae3708bcab774072'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.181105 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6a54f8d1404754ae3708bcab774072'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.191495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6a54f8d1404754ae3708bcab774072'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.194010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6a54f8d1404754ae3708bcab774072'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.195426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f2512bd1-97f9-41f3-8f3b-c5439940b01b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6a54f8d1404754ae3708bcab774072'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.217902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcfb0cb-b6c4-4e50-8a99-50c536ba0df6 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.233935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.235611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.235611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.235611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] PATH_INFO: `/v3/projects/f72ee4751e0747d9af27ca355dde6c1e` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.237045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f72ee4751e0747d9af27ca355dde6c1e)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.239623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.252002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1118/2214] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:10 2022] DELETE /identity/v3/users/7b6a54f8d1404754ae3708bcab774072 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:10.257890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.258265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.258616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.309540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:10.321039 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.415005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f72ee4751e0747d9af27ca355dde6c1e', 'request_id': 'req-10056c12-0402-4932-9780-26d37bd51c5d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.438355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f72ee4751e0747d9af27ca355dde6c1e', 'request_id': 'req-10056c12-0402-4932-9780-26d37bd51c5d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.471617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1119/2215] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:10.472667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f72ee4751e0747d9af27ca355dde6c1e', 'request_id': 'req-10056c12-0402-4932-9780-26d37bd51c5d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.489979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:10.495356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.497329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.510553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] The token cache is being invalidate because project f72ee4751e0747d9af27ca355dde6c1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:10.511740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.513538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-10056c12-0402-4932-9780-26d37bd51c5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:10.516235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1097/2216] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:10 2022] DELETE /identity/v3/projects/f72ee4751e0747d9af27ca355dde6c1e => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:10.521759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a345ba-6b3e-46b1-9231-dcae50ddfaf7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.522063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a345ba-6b3e-46b1-9231-dcae50ddfaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.522325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a345ba-6b3e-46b1-9231-dcae50ddfaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.575845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f1a345ba-6b3e-46b1-9231-dcae50ddfaf7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:10.577073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f1a345ba-6b3e-46b1-9231-dcae50ddfaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.680649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c70b8b-06a6-4e55-88fa-eafe2a6fffb2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.763269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1098/2217] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:10.764905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.765206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.765487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.814149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:10.815435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.854942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1099/2218] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:10.855121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.856444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.860110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.860434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.861385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.868051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:10.873291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2585bd7b-0ee6-4553-9747-8b847fc0f415 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:10.904888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.907053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.913222 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:10.914071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:10.914347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:10.914610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:10.915392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:10.915701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:10.915701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:10.943027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1120/2219] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:10.965085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1100/2220] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:43:10.965905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:10.988158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39338f-eae0-48ec-b8ae-8042650be44f service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.007432 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.008121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.008425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.008681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.017017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.019029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.043377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:11.085572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1121/2221] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:10 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:11.089346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:11.107444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1c2f32-39f5-4b91-871e-712d2ea2f8b9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.236135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.236836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.237131 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.237491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.238124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.240210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.283825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1101/2222] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:11 2022] POST /identity/v3/users => generated 284 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:11.288011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:11.307202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8263d3d1-b331-4d6f-b10a-663af8135e9e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.320092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.320798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.321088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.321375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.322116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.323792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.344204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1122/2223] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:11.347993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:11.361163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd72bad-657a-42ea-917e-0abc71754e9a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.373316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b60179-8965-4376-80ee-9bef8836bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.374015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b60179-8965-4376-80ee-9bef8836bce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.374297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b60179-8965-4376-80ee-9bef8836bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.374568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b60179-8965-4376-80ee-9bef8836bce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.375370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b60179-8965-4376-80ee-9bef8836bce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.384088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b60179-8965-4376-80ee-9bef8836bce1 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.398468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1102/2224] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:11.402528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:11.415547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e5cd1e-aa35-46f3-a151-f1ebc6ce35a0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.433166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.434333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.434787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.435410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] PATH_INFO: `/v3/projects/aebde08df0944391bb35a94e7541f20d/users/ce70428ff29d416c9bcfaca85f7cdf7e/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.471649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aebde08df0944391bb35a94e7541f20d, user_id=ce70428ff29d416c9bcfaca85f7cdf7e, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.474641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.507644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1123/2225] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:11 2022] PUT /identity/v3/projects/aebde08df0944391bb35a94e7541f20d/users/ce70428ff29d416c9bcfaca85f7cdf7e/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:11.517857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.518067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.518349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.582578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] MFA Rules not processed for user `ce70428ff29d416c9bcfaca85f7cdf7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:11.583863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:11.687344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:11.733705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1103/2226] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:11.750092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:11.751962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:11.898070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9acc65-154f-43b7-a468-4ac40ee33749 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:11.918530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:11.919467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:11.919747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:11.920031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:11.961462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:11.963400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:11.995874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1124/2227] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:11 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:13.041319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.044104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:13.129994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd67be1-7e08-4510-b66a-5b28ac7817c7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.138624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.139510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.139796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.140062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] PATH_INFO: `/v3/projects/8333398c61d74e498fed189ddc4484e8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.143987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8333398c61d74e498fed189ddc4484e8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.143987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.202217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8333398c61d74e498fed189ddc4484e8', 'request_id': 'req-f22c8087-9330-4ef1-9a3e-400438a85bf3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.211642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8333398c61d74e498fed189ddc4484e8', 'request_id': 'req-f22c8087-9330-4ef1-9a3e-400438a85bf3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.220981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8333398c61d74e498fed189ddc4484e8', 'request_id': 'req-f22c8087-9330-4ef1-9a3e-400438a85bf3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.240223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] The token cache is being invalidate because project 8333398c61d74e498fed189ddc4484e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:13.241708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.242580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.244298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1104/2228] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:13 2022] DELETE /identity/v3/projects/8333398c61d74e498fed189ddc4484e8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:13.248384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.259686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706e47be-63ec-4840-aacb-34c2e3a7f5ce service glance] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:13.384421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:13.385851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:13.386461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:13.387502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] PATH_INFO: `/v3/users/b83f2c9505374877b5511f4d60ddcd5a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:13.389293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b83f2c9505374877b5511f4d60ddcd5a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:13.391919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:13.474809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b83f2c9505374877b5511f4d60ddcd5a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.489130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b83f2c9505374877b5511f4d60ddcd5a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.495021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:13.507592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b83f2c9505374877b5511f4d60ddcd5a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.508314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b83f2c9505374877b5511f4d60ddcd5a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.510370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6c1aebe4-177d-48bb-ba94-5ea2004b0f0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b83f2c9505374877b5511f4d60ddcd5a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:13.546642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1125/2229] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:13 2022] DELETE /identity/v3/users/b83f2c9505374877b5511f4d60ddcd5a => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:13.974786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:13.976879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:13.994004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22c8087-9330-4ef1-9a3e-400438a85bf3 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.076435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.077339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.077741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.078133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] PATH_INFO: `/v3/projects/df6cd5432d1440fc94cc210c5b74e22a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.079675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6cd5432d1440fc94cc210c5b74e22a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.082664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.115160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:14.164157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6cd5432d1440fc94cc210c5b74e22a', 'request_id': 'req-fde35f46-f037-4ba9-a473-806af726c2cc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:14.174945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6cd5432d1440fc94cc210c5b74e22a', 'request_id': 'req-fde35f46-f037-4ba9-a473-806af726c2cc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:14.192065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6cd5432d1440fc94cc210c5b74e22a', 'request_id': 'req-fde35f46-f037-4ba9-a473-806af726c2cc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:14.208015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] The token cache is being invalidate because project df6cd5432d1440fc94cc210c5b74e22a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:14.208431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:14.209648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-fde35f46-f037-4ba9-a473-806af726c2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:14.211518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1105/2230] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:13 2022] DELETE /identity/v3/projects/df6cd5432d1440fc94cc210c5b74e22a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:14.216744 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.217051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.217318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.256714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:14.257971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:14.384487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1126/2231] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:14.395140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.395390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.395530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.417992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:14.419479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:14.430307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1106/2232] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:14.435518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.451786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334a0cb-3c9d-46c9-a11a-f146d94809c7 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.459075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.460151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.460613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.461067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.463238 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:14.463747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.466315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.483031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1127/2233] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:14.487203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.497416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a09e25-3a17-4330-938e-e9948fad63a3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.506734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.507468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.507757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.508037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.508755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.510682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.542272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1107/2234] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:14 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:14.546359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.590140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb9450d-82b1-4c38-b660-94934970cb61 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.719165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.721018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.721305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.721582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.722311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.727250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.765236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1128/2235] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:14 2022] POST /identity/v3/users => generated 282 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:14.766598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.782459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d9cb3a-8f05-4960-bd2d-873485a4b2a3 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.798982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.800024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.800494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.801503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.802764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.805126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.837536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1108/2236] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:14.838958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.855431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca6c45d-7368-4ade-b386-fb16fb454dc5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.877826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d93ab-c45c-40a0-a3bb-cb46ab169bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.878966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d93ab-c45c-40a0-a3bb-cb46ab169bdc admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.879454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d93ab-c45c-40a0-a3bb-cb46ab169bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.880422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d93ab-c45c-40a0-a3bb-cb46ab169bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.881372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d93ab-c45c-40a0-a3bb-cb46ab169bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.883314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d93ab-c45c-40a0-a3bb-cb46ab169bdc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:14.906507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1129/2237] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:14 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:14.907525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:14.927433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8708e34-1a96-4913-ae8a-14145f9b3306 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:14.944304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:14.945954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:14.946282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:14.946526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] PATH_INFO: `/v3/projects/e7983d12802141d587096a94c55dedea/users/fedba47642424691977bdda750685f2f/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:14.983431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7983d12802141d587096a94c55dedea, user_id=fedba47642424691977bdda750685f2f, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:14.994420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.016169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1109/2238] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:14 2022] PUT /identity/v3/projects/e7983d12802141d587096a94c55dedea/users/fedba47642424691977bdda750685f2f/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:15.030115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.030495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.030893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.109561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] MFA Rules not processed for user `fedba47642424691977bdda750685f2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:15.111809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:15.165911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:15.300569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1130/2239] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:15 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:15.305297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:15.392678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad5ce7-789d-405f-b1e6-2d0bb3a6f025 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:15.400630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:15.401486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.402063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.402883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.403091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:15.406827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.455879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1110/2240] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:15 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:15.457793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:15.469680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f574d8-6878-49b4-861f-64890fdd77b1 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:15.559797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:15.560639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.560973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.561279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.561970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:15.564810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.594941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1131/2241] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:15 2022] POST /identity/v3/users => generated 288 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:15.599432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:15.635719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebc1e3-9e5c-4de1-afd2-9902c7f7df5c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:15.642694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:15.643480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.643768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.644050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.645195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:15.646962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.661154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1111/2242] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:15.666463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:15.678222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85761134-8b04-46d8-8039-9fa628be578c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:15.683607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:15.684859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.685254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.685634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] PATH_INFO: `/v3/projects/4b2aab012cc04966822697bd2d368be6/users/e897007bd6bf40d4a96d6d0afbd384c3/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.723802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2aab012cc04966822697bd2d368be6, user_id=e897007bd6bf40d4a96d6d0afbd384c3, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:15.726949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.744036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1132/2243] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:15 2022] PUT /identity/v3/projects/4b2aab012cc04966822697bd2d368be6/users/e897007bd6bf40d4a96d6d0afbd384c3/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:15.747769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:15.760750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103980c-7265-4e18-8cff-d324ed769577 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:15.780165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:15.857927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c90c75-e04a-461a-94cf-c8a55ef97712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:15.858774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c90c75-e04a-461a-94cf-c8a55ef97712 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.859058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c90c75-e04a-461a-94cf-c8a55ef97712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.859378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c90c75-e04a-461a-94cf-c8a55ef97712 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.861139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c90c75-e04a-461a-94cf-c8a55ef97712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:15.864911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c90c75-e04a-461a-94cf-c8a55ef97712 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.881777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1112/2244] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:15 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:15.886067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:15.897703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da352c5-c196-4f0b-971d-fa00d1ee4dcd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:15.902814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:15.903865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.904228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.905021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] PATH_INFO: `/v3/projects/4b2aab012cc04966822697bd2d368be6/users/e897007bd6bf40d4a96d6d0afbd384c3/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:15.906837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2aab012cc04966822697bd2d368be6, user_id=e897007bd6bf40d4a96d6d0afbd384c3, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:15.909740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:15.932102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1133/2245] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:15 2022] PUT /identity/v3/projects/4b2aab012cc04966822697bd2d368be6/users/e897007bd6bf40d4a96d6d0afbd384c3/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:15.944680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:15.945320 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:15.945715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:16.001212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] MFA Rules not processed for user `e897007bd6bf40d4a96d6d0afbd384c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:16.004842 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:16.216665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1113/2246] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:15 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:16.229664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:16.233736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:16.380848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e910c815-a29c-4469-a45b-480ac7931550 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:16.385511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d130b7-6f9b-4a20-978e-081a9acab2d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:16.386705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d130b7-6f9b-4a20-978e-081a9acab2d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:16.387080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d130b7-6f9b-4a20-978e-081a9acab2d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:16.387499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d130b7-6f9b-4a20-978e-081a9acab2d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:16.413484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d130b7-6f9b-4a20-978e-081a9acab2d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:16.420481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d130b7-6f9b-4a20-978e-081a9acab2d9 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:16.436461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1134/2247] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:16.545299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:16.560071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f508f09-e0ec-44a8-b243-e3d41e52ad69 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:16.564830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6291-788e-4187-b441-d62021957216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:16.565658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6291-788e-4187-b441-d62021957216 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:16.566047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6291-788e-4187-b441-d62021957216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:16.566986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69be6291-788e-4187-b441-d62021957216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:16.682297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be6291-788e-4187-b441-d62021957216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:16.685950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69be6291-788e-4187-b441-d62021957216 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:16.703723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1114/2248] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:16 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:17.475073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09179ce7-232f-43bb-9893-b5cab2142391 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:17.476002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09179ce7-232f-43bb-9893-b5cab2142391 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:17.476920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09179ce7-232f-43bb-9893-b5cab2142391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:17.520212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-09179ce7-232f-43bb-9893-b5cab2142391 None None] MFA Rules not processed for user `3cb45eb7f35848ea86cffb221f64f1dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:17.521930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-09179ce7-232f-43bb-9893-b5cab2142391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:17.543435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:17.630778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1135/2249] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:17 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:17.637464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6291-788e-4187-b441-d62021957216 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:17.639532 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-69be6291-788e-4187-b441-d62021957216 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:17.691489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:17.723813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69be6291-788e-4187-b441-d62021957216 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:17.728988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:17.729851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:17.730295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:17.730753 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:17.732791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:17.809900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:17.812206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:17.825401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1115/2250] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:18.021926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:18.022215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:18.022496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:18.043302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:18.044497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:18.113035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1136/2251] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:18.189782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:18.191383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:18.371672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1accab02-4fa0-4338-af2d-3f0121e27ba5 service glance] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:18.382343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:18.383553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:18.384024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:18.384334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:18.396452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:18.397001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:18.407661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1116/2252] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:20.252985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:20.532022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12241281-dd3d-4980-ad6d-5cae0ab0bcc4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:20.557132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:20.558557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:20.559113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:20.559749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] PATH_INFO: `/v3/users/fedba47642424691977bdda750685f2f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:20.563135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fedba47642424691977bdda750685f2f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:20.571780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:20.772199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fedba47642424691977bdda750685f2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:20.784555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fedba47642424691977bdda750685f2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:20.789310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:20.796140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fedba47642424691977bdda750685f2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:20.798663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fedba47642424691977bdda750685f2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:20.800138 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-4d2bc3b6-5d3d-4001-ad93-5d463935248a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fedba47642424691977bdda750685f2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:20.844238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:20.845088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1137/2253] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:20 2022] DELETE /identity/v3/users/fedba47642424691977bdda750685f2f => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:20.849581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:20.849766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:20.850179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:20.850886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:20.885853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33129bc5-5d9c-425e-9159-876169624d1b service swift] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:20.887868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:20.888758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:21.042618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:21.045413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:21.045703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:21.046173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] PATH_INFO: `/v3/users/f83e72cd0d7e47b6bbfc4ffe64b71922` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:21.050273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f83e72cd0d7e47b6bbfc4ffe64b71922)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:21.053001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:21.081341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1138/2254] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:21.086136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:21.099450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:21.101006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:21.157092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f83e72cd0d7e47b6bbfc4ffe64b71922'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:21.176644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f83e72cd0d7e47b6bbfc4ffe64b71922'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:21.197085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f83e72cd0d7e47b6bbfc4ffe64b71922'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:21.207038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f83e72cd0d7e47b6bbfc4ffe64b71922'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:21.216206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f83e72cd0d7e47b6bbfc4ffe64b71922'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:21.225671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1117/2255] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:20 2022] DELETE /identity/v3/users/f83e72cd0d7e47b6bbfc4ffe64b71922 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:21.236918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09676fcb-f9ca-43e7-a157-797aef37ec8f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:21.322467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:21.323263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:21.323566 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:21.324181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:21.330221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:21.384095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:21.414526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:21.416233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcefcfc-d222-4045-8184-20ae7d666d56 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:21.429310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1139/2256] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:21.787268 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:21.788907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:21.837112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cfc0c4-5cae-4358-b880-ac1a1b034f4f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:21.859538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:21.860421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:21.860710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:21.860978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] PATH_INFO: `/v3/projects/6b1d15dc76094127a7f84c79dbb3053c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:21.864840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1d15dc76094127a7f84c79dbb3053c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:21.870750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:21.989119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1d15dc76094127a7f84c79dbb3053c', 'request_id': 'req-5b55693a-d243-46b4-8cd8-35eb65f011ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.003146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1d15dc76094127a7f84c79dbb3053c', 'request_id': 'req-5b55693a-d243-46b4-8cd8-35eb65f011ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.015933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1d15dc76094127a7f84c79dbb3053c', 'request_id': 'req-5b55693a-d243-46b4-8cd8-35eb65f011ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.057477 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] The token cache is being invalidate because project 6b1d15dc76094127a7f84c79dbb3053c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:22.067863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.070836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.076102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1118/2257] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:21 2022] DELETE /identity/v3/projects/6b1d15dc76094127a7f84c79dbb3053c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:22.080973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f00c-3284-4e4b-a64e-21f3280682b6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.081286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f00c-3284-4e4b-a64e-21f3280682b6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.081569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a19f00c-3284-4e4b-a64e-21f3280682b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.118062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4a19f00c-3284-4e4b-a64e-21f3280682b6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:22.119500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4a19f00c-3284-4e4b-a64e-21f3280682b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:22.297661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.299203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1140/2258] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:22.306648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:22.309699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.310395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.311339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.325296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b55693a-d243-46b4-8cd8-35eb65f011ea admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.336670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.337965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.338345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.338575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] PATH_INFO: `/v3/projects/e7983d12802141d587096a94c55dedea` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.341126 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7983d12802141d587096a94c55dedea)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.364376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.367206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:22.369797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:22.399790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1141/2259] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:22.401118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.412827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53ce66-ce7c-43b9-adcd-260e96dc7d92 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.420736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.421598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.422167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.422454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.423399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:22.423766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.425665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.445963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1142/2260] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:22.446977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.466167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2934d384-0e54-404c-9f14-84fcb26a9297 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.469708 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7983d12802141d587096a94c55dedea', 'request_id': 'req-699d6dcb-864d-4886-861e-f10076d8d2b0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.489325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7983d12802141d587096a94c55dedea', 'request_id': 'req-699d6dcb-864d-4886-861e-f10076d8d2b0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.503301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7983d12802141d587096a94c55dedea', 'request_id': 'req-699d6dcb-864d-4886-861e-f10076d8d2b0'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.530838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] The token cache is being invalidate because project e7983d12802141d587096a94c55dedea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:22.531363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.532043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.536409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1119/2261] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:22 2022] DELETE /identity/v3/projects/e7983d12802141d587096a94c55dedea => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:22.544008 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.545770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:22.558336 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=21,keystone:SELECT=76,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:22.604499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.605288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.605603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.606067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.606767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.608853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.658645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1143/2262] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:22 2022] POST /identity/v3/projects => generated 373 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:22.663230 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.667235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:22.669130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d6dcb-864d-4886-861e-f10076d8d2b0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.772343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:22.798749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80eb06a9-4651-427f-8a69-9140188b5d51 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.803940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.804332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.804633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.804919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.805208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.805591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.805890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.806041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.806656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] PATH_INFO: `/v3/users/e897007bd6bf40d4a96d6d0afbd384c3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.809695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.810207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e897007bd6bf40d4a96d6d0afbd384c3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.812360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.847082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1144/2263] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:22 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:22.857798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.875676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8939bc6-c5fc-4347-a86f-d4c518efddbe admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.885176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.886271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.886573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.886833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.888136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.897113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e897007bd6bf40d4a96d6d0afbd384c3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.908123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.911711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e897007bd6bf40d4a96d6d0afbd384c3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.937259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1145/2264] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:22.940985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:22.949905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e897007bd6bf40d4a96d6d0afbd384c3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.964813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb39da7f-2cba-40d4-900b-219cd95857a6 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:22.967572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e897007bd6bf40d4a96d6d0afbd384c3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.969067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e897007bd6bf40d4a96d6d0afbd384c3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:22.978276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf43110-0346-4659-a3bd-1a0ecf8fe40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:22.979464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43110-0346-4659-a3bd-1a0ecf8fe40d admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:22.981732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43110-0346-4659-a3bd-1a0ecf8fe40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:22.982144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf43110-0346-4659-a3bd-1a0ecf8fe40d admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:22.984457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf43110-0346-4659-a3bd-1a0ecf8fe40d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:22.986193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf43110-0346-4659-a3bd-1a0ecf8fe40d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:22.986544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1120/2265] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:22 2022] DELETE /identity/v3/users/e897007bd6bf40d4a96d6d0afbd384c3 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:23.005871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1146/2266] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:23.007408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:23.009004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:23.095890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:23.187613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb52bb0-8741-4c94-bbef-c5945a6eaffe admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:23.200082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:23.201134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:23.201525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:23.201904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] PATH_INFO: `/v3/projects/10c723000139447ab45607f1e57e939e/users/5426fcab1dcd4eb39dcde73c7be2988a/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:23.247906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10c723000139447ab45607f1e57e939e, user_id=5426fcab1dcd4eb39dcde73c7be2988a, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:23.252397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:23.270240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1121/2267] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:23 2022] PUT /identity/v3/projects/10c723000139447ab45607f1e57e939e/users/5426fcab1dcd4eb39dcde73c7be2988a/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:23.281232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:23.281710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:23.282124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:23.321087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] MFA Rules not processed for user `5426fcab1dcd4eb39dcde73c7be2988a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:23.322444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:23.448444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1147/2268] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:23.453803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:23.680738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996dc980-6677-4026-b0d7-985220da9469 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:23.690133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:23.692132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:23.698245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:23.698411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:23.698528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:23.699518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:23.701716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:23.710774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:23.780016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589f9b2-83ee-4a6e-b208-f7c1b8d5d4a1 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:23.805450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:23.806347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:23.806623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:23.806898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] PATH_INFO: `/v3/projects/4b2aab012cc04966822697bd2d368be6` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:23.815944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2aab012cc04966822697bd2d368be6)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:23.830496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:23.861162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1122/2269] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:23 2022] POST /identity/v3/projects => generated 373 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:23.862520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:23.988204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2aab012cc04966822697bd2d368be6', 'request_id': 'req-f91353b7-130b-41c7-a706-4005499daabb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:24.008632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e472894-e016-43ce-850c-b4b794e8a7bb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.012362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2aab012cc04966822697bd2d368be6', 'request_id': 'req-f91353b7-130b-41c7-a706-4005499daabb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:24.022971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2aab012cc04966822697bd2d368be6', 'request_id': 'req-f91353b7-130b-41c7-a706-4005499daabb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:24.054080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] The token cache is being invalidate because project 4b2aab012cc04966822697bd2d368be6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:24.054380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:24.056210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f91353b7-130b-41c7-a706-4005499daabb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:24.058541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1148/2270] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:23 2022] DELETE /identity/v3/projects/4b2aab012cc04966822697bd2d368be6 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:24.069096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17406896-bfac-43b8-8704-1f6b1759f0b4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.069378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17406896-bfac-43b8-8704-1f6b1759f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.069659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17406896-bfac-43b8-8704-1f6b1759f0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.076081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:24.095051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-17406896-bfac-43b8-8704-1f6b1759f0b4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:24.096422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-17406896-bfac-43b8-8704-1f6b1759f0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.101904 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.103113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.103808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.104290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.105270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.107419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.116667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1149/2271] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:24.118402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.118822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.119167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.142516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:24.143880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.148409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1123/2272] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:23 2022] POST /identity/v3/users => generated 293 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:24.151744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.154278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.169250 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1150/2273] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:24.172525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.184621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bf1c89-80a8-4690-b78e-654c7b6193c9 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.190109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.190798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.191148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.191492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.191893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3094c09d-7905-4311-ab6b-bc6478b76b2b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.192236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:24.192544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.194334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.208461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.209407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.210015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.210441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.211376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.213662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.229238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1124/2274] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:24.234285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1151/2275] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:24.236808 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.237072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.250745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:24.263072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c508c6-cafd-4093-b91a-3cc560c11065 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.266693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8dd79c-555e-41e7-81c3-b0a3f320d0d5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.275474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.276417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.276711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.276994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] PATH_INFO: `/v3/projects/59db27771eba4e0da44661434df9279f/users/ca5a481b1d4f4c778bf34c0ee09210ea/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.277359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.278187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.278622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.279117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.279983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.283560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.303956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59db27771eba4e0da44661434df9279f, user_id=ca5a481b1d4f4c778bf34c0ee09210ea, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.309134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.327204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1152/2276] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:24.333036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.333474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1125/2277] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:24 2022] PUT /identity/v3/projects/59db27771eba4e0da44661434df9279f/users/ca5a481b1d4f4c778bf34c0ee09210ea/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:24.336392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.426277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69211f8-14d8-4e2e-a3ad-34307062223e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.432782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346d9d5-786a-406e-8502-78d1289966cb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.433384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.434160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.434448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.434730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.435525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.437387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.439545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.440431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.440802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.442359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.443187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.446192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.451184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1126/2278] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:24.455232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.469421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f0d8fc-0cff-426c-932d-66fff66ed0eb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.478190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918c94e-6992-4a37-8db8-5ea9d2c5f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.479619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c94e-6992-4a37-8db8-5ea9d2c5f032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.479917 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c94e-6992-4a37-8db8-5ea9d2c5f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.480459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918c94e-6992-4a37-8db8-5ea9d2c5f032 admin admin] PATH_INFO: `/v3/projects/59db27771eba4e0da44661434df9279f/users/ca5a481b1d4f4c778bf34c0ee09210ea/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.482986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918c94e-6992-4a37-8db8-5ea9d2c5f032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59db27771eba4e0da44661434df9279f, user_id=ca5a481b1d4f4c778bf34c0ee09210ea, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.486147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918c94e-6992-4a37-8db8-5ea9d2c5f032 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.491394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1153/2279] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/users => generated 293 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:24.494323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.509282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1127/2280] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:24 2022] PUT /identity/v3/projects/59db27771eba4e0da44661434df9279f/users/ca5a481b1d4f4c778bf34c0ee09210ea/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:24.512913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1d90f8-0a25-4420-bdc2-c4d0feb96a9f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.520722 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.521027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.521311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.583553 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] MFA Rules not processed for user `ca5a481b1d4f4c778bf34c0ee09210ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:24.584773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.607996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.608698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.608993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.609292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.610047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.611702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.625826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1154/2281] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:24.631244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.658423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82ab7-f62e-40c3-8727-c184a9b94649 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.664777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.665757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.666145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.666420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.667802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.669563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.682570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1128/2282] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:24.686793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1155/2283] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:24.687206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.694888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:24.696895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.705716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf443a61-6f03-415d-8aa6-51a19338eaa5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.714118 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5700a6-f0f7-49ba-a05f-68a5e9b3aa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.714938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5700a6-f0f7-49ba-a05f-68a5e9b3aa5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.715616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5700a6-f0f7-49ba-a05f-68a5e9b3aa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.715912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5700a6-f0f7-49ba-a05f-68a5e9b3aa5a admin admin] PATH_INFO: `/v3/projects/0e05f049758043cb9c0c1095bc4e0b95/users/9e304a7a988747b68265df63e05ec9a5/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.720410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:24.743342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5700a6-f0f7-49ba-a05f-68a5e9b3aa5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e05f049758043cb9c0c1095bc4e0b95, user_id=9e304a7a988747b68265df63e05ec9a5, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:24.747069 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5700a6-f0f7-49ba-a05f-68a5e9b3aa5a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:24.783251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1129/2284] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:24 2022] PUT /identity/v3/projects/0e05f049758043cb9c0c1095bc4e0b95/users/9e304a7a988747b68265df63e05ec9a5/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:24.791573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c74753-6457-48e9-93f8-77945280e80b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.792111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c74753-6457-48e9-93f8-77945280e80b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.792398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c74753-6457-48e9-93f8-77945280e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.795503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51da990-7a54-4c62-b013-16059f71a4cd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:24.806724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:24.842845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-82c74753-6457-48e9-93f8-77945280e80b None None] MFA Rules not processed for user `9e304a7a988747b68265df63e05ec9a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:24.844745 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-82c74753-6457-48e9-93f8-77945280e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:24.870791 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:24.871567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:24.871864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:24.872151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:24.939066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1130/2285] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:24.943331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c74753-6457-48e9-93f8-77945280e80b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.020926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.022407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.029326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c74753-6457-48e9-93f8-77945280e80b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.033835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1156/2286] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:25.034764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.035522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.035867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.036214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.037059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.039325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.072689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1131/2287] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:24 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:25.078734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.109785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae224824-da54-4997-ae73-bac6d04edfdc service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.225827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.226377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.226669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.226946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.227701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.229655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.259511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1157/2288] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:25 2022] POST /identity/v3/users => generated 294 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:25.264649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.288675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7437d239-a642-4961-b84e-834b79edaf40 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.312611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.313474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.313829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.314180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.316106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.318285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.346082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1132/2289] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:25.351320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.376066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a90c6da-bbf0-4bff-85be-bab04db399a4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.382471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dac6ee-9dd9-4e80-ab9b-d9721189aa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.383151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dac6ee-9dd9-4e80-ab9b-d9721189aa0a admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.383471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dac6ee-9dd9-4e80-ab9b-d9721189aa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.383756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dac6ee-9dd9-4e80-ab9b-d9721189aa0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.384498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dac6ee-9dd9-4e80-ab9b-d9721189aa0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.386127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dac6ee-9dd9-4e80-ab9b-d9721189aa0a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.417315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1158/2290] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:25 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:25.417884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.432725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a797b0-0801-415c-9e1d-98f7b20a7248 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.439328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.440331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.440713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.441134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] PATH_INFO: `/v3/projects/273ca832d6f1464cbe2f428b06e28019/users/41d98595894a443fa0a9f358af0b78aa/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.460716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=273ca832d6f1464cbe2f428b06e28019, user_id=41d98595894a443fa0a9f358af0b78aa, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.463928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.498188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1133/2291] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:25 2022] PUT /identity/v3/projects/273ca832d6f1464cbe2f428b06e28019/users/41d98595894a443fa0a9f358af0b78aa/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:25.509459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.510169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.510169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.556090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] MFA Rules not processed for user `41d98595894a443fa0a9f358af0b78aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:25.559559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:25.641758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.655310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:25.680577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:25.751289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1159/2292] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:25.752317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.760021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af286f-3917-4bb2-b286-e9b3c1530446 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.772606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c79b76-5027-44dc-9fa5-04600dbbd1e4 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:25.773643 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.774536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.774818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.775301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] PATH_INFO: `/v3/users/5426fcab1dcd4eb39dcde73c7be2988a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.776897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5426fcab1dcd4eb39dcde73c7be2988a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.779304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.789050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:25.789706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.790032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.790340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.791113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:25.793705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:25.852129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1160/2293] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:25 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:25.857020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:25.873582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5426fcab1dcd4eb39dcde73c7be2988a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:25.887139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5426fcab1dcd4eb39dcde73c7be2988a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:25.905104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5426fcab1dcd4eb39dcde73c7be2988a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:25.916121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5426fcab1dcd4eb39dcde73c7be2988a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:25.917133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-622faa80-60a8-493b-899c-e6c70e1caeb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5426fcab1dcd4eb39dcde73c7be2988a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:25.927229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1134/2294] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:25 2022] DELETE /identity/v3/users/5426fcab1dcd4eb39dcde73c7be2988a => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:25.932600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:25.932890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:25.933169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:25.957359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:25.960014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:25.975469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a39de3-4d0f-4dea-8d6a-23cc4ff32554 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.056830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1135/2295] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:26.069088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.070641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.100106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.100806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.101251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.101539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.102381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.104902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.145783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1161/2296] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:25 2022] POST /identity/v3/users => generated 300 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:26.146831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.156757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:26.157218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.175378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520d83c-2d7a-44bc-b46f-7cff7fed3b6c None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.181598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.182525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.183695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.184231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.188848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c95dd7-367d-4e00-b6b8-146bb12d47d8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.194390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.195165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.195549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.195874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.195991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.196701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.197337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.198541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.209670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1136/2297] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:26.222281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1162/2298] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:26.224184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.234441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af34d6da-ec1d-42ae-9890-4698d2412203 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.243247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.244690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.245220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.245696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] PATH_INFO: `/v3/projects/4d40bee618b440ce93282bf5a0c71c2f/users/d3aff2964db541a18e49f584f66c6795/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.263652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d40bee618b440ce93282bf5a0c71c2f, user_id=d3aff2964db541a18e49f584f66c6795, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.268121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.289536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1137/2299] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:26 2022] PUT /identity/v3/projects/4d40bee618b440ce93282bf5a0c71c2f/users/d3aff2964db541a18e49f584f66c6795/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:26.293848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.308691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed67ac07-fc07-46d6-b955-497edacd6223 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.346613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:26.382738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610b6f2f-8efc-4eba-a144-a07e6403de1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.383584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6f2f-8efc-4eba-a144-a07e6403de1f admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.383972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6f2f-8efc-4eba-a144-a07e6403de1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.384352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610b6f2f-8efc-4eba-a144-a07e6403de1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.385301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6f2f-8efc-4eba-a144-a07e6403de1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.388082 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610b6f2f-8efc-4eba-a144-a07e6403de1f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.415317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1163/2300] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:26 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:26.420673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.436411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa09dde1-b967-4283-9759-3db674b0984a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.450224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.451278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.451591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.452068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] PATH_INFO: `/v3/projects/4d40bee618b440ce93282bf5a0c71c2f/users/d3aff2964db541a18e49f584f66c6795/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.454202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d40bee618b440ce93282bf5a0c71c2f, user_id=d3aff2964db541a18e49f584f66c6795, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.457612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.483760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1138/2301] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:26 2022] PUT /identity/v3/projects/4d40bee618b440ce93282bf5a0c71c2f/users/d3aff2964db541a18e49f584f66c6795/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:26.485055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.485414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.485699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.528909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] MFA Rules not processed for user `d3aff2964db541a18e49f584f66c6795`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:26.530151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.565478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.568075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.645212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1164/2302] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:26.652829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.655253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e887d6c5-f580-4933-8980-97c3b5b6deab admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.655517 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.662017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.663953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.664296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.665047 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] PATH_INFO: `/v3/projects/10c723000139447ab45607f1e57e939e` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.665826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10c723000139447ab45607f1e57e939e)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.668119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.732465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d670a6-d7bd-4535-b0da-c3887ba396d0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.735454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10c723000139447ab45607f1e57e939e', 'request_id': 'req-6b995b83-07f6-488a-88e6-299de34f1030'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:26.745376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10c723000139447ab45607f1e57e939e', 'request_id': 'req-6b995b83-07f6-488a-88e6-299de34f1030'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:26.754350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10c723000139447ab45607f1e57e939e', 'request_id': 'req-6b995b83-07f6-488a-88e6-299de34f1030'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:26.766891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] The token cache is being invalidate because project 10c723000139447ab45607f1e57e939e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:26.767224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:26.767930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:26.769660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1139/2303] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:26 2022] DELETE /identity/v3/projects/10c723000139447ab45607f1e57e939e => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:26.773449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:26.774816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.805341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.822114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.822343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.822633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.829692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:26.845106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:26.882679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b995b83-07f6-488a-88e6-299de34f1030 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:26.888367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:26.889695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:26.890029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:26.890312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] PATH_INFO: `/v3/users/ca5a481b1d4f4c778bf34c0ee09210ea` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:26.891768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca5a481b1d4f4c778bf34c0ee09210ea)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.893236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:26.896292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:26.980317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:26.981048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca5a481b1d4f4c778bf34c0ee09210ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:26.985368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:27.001949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca5a481b1d4f4c778bf34c0ee09210ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:27.009076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1165/2304] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:27.029158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca5a481b1d4f4c778bf34c0ee09210ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:27.038980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca5a481b1d4f4c778bf34c0ee09210ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:27.039875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca5a481b1d4f4c778bf34c0ee09210ea'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:27.051463 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1140/2305] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:26 2022] DELETE /identity/v3/users/ca5a481b1d4f4c778bf34c0ee09210ea => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:27.773966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:27.776631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:27.790646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafb4491-f88b-40e3-a26f-fbe4a64f63cd service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:27.856947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:27.857858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:27.858256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:27.858699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:27.861641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:27.945176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:27.946719 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acc33f9-0441-4c08-89d3-c4bfe7eb2df2 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:27.958566 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1166/2306] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:28.038280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.040285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:28.191851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49920113-a9c1-477e-8eca-49c607745a3c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.197369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.198436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.198912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.199427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] PATH_INFO: `/v3/projects/59db27771eba4e0da44661434df9279f` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.201823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59db27771eba4e0da44661434df9279f)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.205249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:28.297339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59db27771eba4e0da44661434df9279f', 'request_id': 'req-98f57465-1302-44a6-b1b8-6e632621de1d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:28.309450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59db27771eba4e0da44661434df9279f', 'request_id': 'req-98f57465-1302-44a6-b1b8-6e632621de1d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:28.318590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59db27771eba4e0da44661434df9279f', 'request_id': 'req-98f57465-1302-44a6-b1b8-6e632621de1d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:28.320019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:28.337235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] The token cache is being invalidate because project 59db27771eba4e0da44661434df9279f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:28.337235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:28.338024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98f57465-1302-44a6-b1b8-6e632621de1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:28.340113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1141/2307] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:28 2022] DELETE /identity/v3/projects/59db27771eba4e0da44661434df9279f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:28.344819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.345684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.346123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.375620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:28.377014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:28.490117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1167/2308] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:28.498105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.499001 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.499542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.524613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:28.525797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:28.548981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.549613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1142/2309] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:28.564988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085d7e70-918f-4d20-a876-bb88527b80ae None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.584645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.585382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.585713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.586036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.588090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:28.588090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.588907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:28.618381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1168/2310] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:43:28.622145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.643407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6e80-42dc-413f-a8c7-659532f19f68 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.649445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.650234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.650622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.650995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.651820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.653955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:28.695539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1143/2311] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:28 2022] POST /identity/v3/projects => generated 367 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:28.699666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.707444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:28.714644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e1726-f933-4fae-9f5e-11fa24c9d340 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.788488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.789347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.789734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.790132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.790946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.793329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:28.841396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.847338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1169/2312] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:28 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:28.856067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e6376f-f7df-438e-b055-54fbcdce3265 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.861892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.862680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.862970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.863315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.864109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.866589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:28.878308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1144/2313] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:28.882324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.899023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afef5f-7501-4bc5-b979-ef6e5724e39b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.906216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5503641-e676-475a-9d75-8b202be6acec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.907157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503641-e676-475a-9d75-8b202be6acec admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.907542 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503641-e676-475a-9d75-8b202be6acec admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.907877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503641-e676-475a-9d75-8b202be6acec admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.908691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5503641-e676-475a-9d75-8b202be6acec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.910474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5503641-e676-475a-9d75-8b202be6acec admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:28.931264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1170/2314] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:28.936302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:28.959497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a14671f-c6d8-4175-bc10-e9e639441b99 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:28.966397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:28.967513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:28.968015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:28.968359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] PATH_INFO: `/v3/projects/afb9becaa4404cf885f80466582421d7/users/3099636332114537893c0321b2656365/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:28.985317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afb9becaa4404cf885f80466582421d7, user_id=3099636332114537893c0321b2656365, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:28.989006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.014719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1145/2315] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:28 2022] PUT /identity/v3/projects/afb9becaa4404cf885f80466582421d7/users/3099636332114537893c0321b2656365/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:29.015984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.016302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.016581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.055347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] MFA Rules not processed for user `3099636332114537893c0321b2656365`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:29.070955 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:29.161191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1171/2316] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:29.167679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.252730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039db442-d395-463f-8d1f-6555d454d611 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.258039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.258714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.259002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.259439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.260144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.263467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.326613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1146/2317] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:29 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:29.327775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.339270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d3b3c5-7f91-459a-8c55-45c79935f0fc None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.405475 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.406368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.406653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.406936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.407673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.409590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.427713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:29.434280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1172/2318] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:29 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:29.439386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.451672 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e97160d-efd3-41eb-9b77-a06632bdbb1b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.459742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.460427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.460716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.461056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.461827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.463494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.480384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.480762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1147/2319] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:29.493132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400bf635-82d6-461f-974c-ed0cd1f5433d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.502181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.504136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.504136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.504434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] PATH_INFO: `/v3/projects/fa646cb5798349758ede8814c56129cc/users/a9de35ce67fe43f398a58429811d3905/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.523900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa646cb5798349758ede8814c56129cc, user_id=a9de35ce67fe43f398a58429811d3905, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.527198 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.555306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1173/2320] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:29 2022] PUT /identity/v3/projects/fa646cb5798349758ede8814c56129cc/users/a9de35ce67fe43f398a58429811d3905/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:29.555746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.570830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1b9f9-f889-4765-9d9a-d5cefb8b3ce2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.655496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634e26f-5682-4b84-9a6b-d02280e47f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.656294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634e26f-5682-4b84-9a6b-d02280e47f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.656680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634e26f-5682-4b84-9a6b-d02280e47f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.657063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634e26f-5682-4b84-9a6b-d02280e47f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.657902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634e26f-5682-4b84-9a6b-d02280e47f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.659879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634e26f-5682-4b84-9a6b-d02280e47f9a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.662259 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:29.678322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1148/2321] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:29 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:29.679606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.694374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eed7ac-a4b2-4e39-89b8-df48753c4d64 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.699865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.700864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.701256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.701641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] PATH_INFO: `/v3/projects/fa646cb5798349758ede8814c56129cc/users/a9de35ce67fe43f398a58429811d3905/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.704023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa646cb5798349758ede8814c56129cc, user_id=a9de35ce67fe43f398a58429811d3905, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:29.706824 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:29.733305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1174/2322] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:29 2022] PUT /identity/v3/projects/fa646cb5798349758ede8814c56129cc/users/a9de35ce67fe43f398a58429811d3905/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:29.736603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.737128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.737534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.784647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] MFA Rules not processed for user `a9de35ce67fe43f398a58429811d3905`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:29.786811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:29.892762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1149/2323] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:29 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:29.906736 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:29.908531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:29.982309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2724a4-5668-4444-9801-b4a398cb3634 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:29.986352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e078c-45bf-403d-af0b-fb83772e83fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:29.987012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e078c-45bf-403d-af0b-fb83772e83fb service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:29.987426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e078c-45bf-403d-af0b-fb83772e83fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:29.987806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e078c-45bf-403d-af0b-fb83772e83fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:29.998825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e078c-45bf-403d-af0b-fb83772e83fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:30.000547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e078c-45bf-403d-af0b-fb83772e83fb service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:30.012363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1175/2324] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:29 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:30.067651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:30.069068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:30.100580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2e72b-b4ed-483b-a25f-c00949ff6dc3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:30.104729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:30.105629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:30.106096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:30.106642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:30.188868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:30.190475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:30.215028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1150/2325] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:32.386415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92d629-e301-4222-925a-aa20829e0596 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.387047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92d629-e301-4222-925a-aa20829e0596 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.387360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a92d629-e301-4222-925a-aa20829e0596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.433258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7a92d629-e301-4222-925a-aa20829e0596 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:32.434872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7a92d629-e301-4222-925a-aa20829e0596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.579246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1176/2326] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:32.615474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:32.618526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:32.939251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5d6a14-ee7c-4857-8f05-d139b3118f93 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:32.944827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:32.945491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:32.945785 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:32.946270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:32.957587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:32.958315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:32.968269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1151/2327] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:33.780125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b3ab3-7927-43cb-9a36-fc4bce508b51 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.780606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b3ab3-7927-43cb-9a36-fc4bce508b51 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.780697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b3ab3-7927-43cb-9a36-fc4bce508b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.805945 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-933b3ab3-7927-43cb-9a36-fc4bce508b51 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:33.807204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-933b3ab3-7927-43cb-9a36-fc4bce508b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:33.814728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1177/2328] 10.222.0.43 () {58 vars in 967 bytes} [Fri Jun 24 08:43:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:33.829537 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:33.842160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fa7553-72f6-4719-9457-8cc0db6e075e service swift] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:33.848119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2522683-1df7-4fcb-a9ae-3262bc6eeb0b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:33.848764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2522683-1df7-4fcb-a9ae-3262bc6eeb0b service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:33.849044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2522683-1df7-4fcb-a9ae-3262bc6eeb0b service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:33.849314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2522683-1df7-4fcb-a9ae-3262bc6eeb0b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:33.856893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:33.860396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2522683-1df7-4fcb-a9ae-3262bc6eeb0b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:33.862670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2522683-1df7-4fcb-a9ae-3262bc6eeb0b service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:33.879316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1152/2329] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:34.014798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.015168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.015531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.055248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] MFA Rules not processed for user `4dbbf627f3bf4ffb87c6e23fac0b5002`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:34.056633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:34.137012 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1178/2330] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:34.200595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.200938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.201317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.233334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:34.236492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:34.260193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1153/2331] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:34.278745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:34.292402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a4eb1c-95e1-4796-9239-7b5d92230471 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:34.299560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc845e97-2017-459c-bd1e-53b6758ca839 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:34.300435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc845e97-2017-459c-bd1e-53b6758ca839 service swift] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.301690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc845e97-2017-459c-bd1e-53b6758ca839 service swift] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.302135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc845e97-2017-459c-bd1e-53b6758ca839 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.311351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:34.318682 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc845e97-2017-459c-bd1e-53b6758ca839 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:34.322576 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc845e97-2017-459c-bd1e-53b6758ca839 service swift] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:34.333653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1179/2332] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:34.367523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:34.448043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91083d-ac17-49a8-9dfa-fa390cf19584 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:34.453538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:34.454339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.454622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.454900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] PATH_INFO: `/v3/users/3099636332114537893c0321b2656365` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.456149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3099636332114537893c0321b2656365)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:34.458702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:34.522788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3099636332114537893c0321b2656365'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:34.535656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3099636332114537893c0321b2656365'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:34.554095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3099636332114537893c0321b2656365'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:34.566021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3099636332114537893c0321b2656365'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:34.567456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3099636332114537893c0321b2656365'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:34.576059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1154/2333] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:34 2022] DELETE /identity/v3/users/3099636332114537893c0321b2656365 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:34.581484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe320049-af84-428c-8f6a-152558df4d91 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.582819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe320049-af84-428c-8f6a-152558df4d91 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.583594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe320049-af84-428c-8f6a-152558df4d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.618878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-fe320049-af84-428c-8f6a-152558df4d91 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:34.623606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fe320049-af84-428c-8f6a-152558df4d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:34.716575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1180/2334] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:34.726584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:34.728436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:34.782713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.783255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.783369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.798890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0314f17d-88cb-430b-9583-58380448487b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:34.802632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:34.803331 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:34.803460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:34.803666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:34.803876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:34.804918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:34.813793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:34.815381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:34.832792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1155/2335] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:34.902026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1181/2336] 10.222.0.43 () {58 vars in 967 bytes} [Fri Jun 24 08:43:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:34.924990 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:34.927183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:34.948512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:35.054519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61057836-a248-43f9-86d7-7539022dd66b service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:35.075255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:35.078352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:35.079528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:35.079987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:35.090565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:35.092732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:35.119915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1156/2337] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:35.597941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:35.600679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:35.695201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc7f974-477a-4988-b434-eb876f445d10 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:35.699804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:35.701201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:35.711084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:35.711751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:35.712059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:35.712375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:35.714981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:35.727534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e02329-9ecc-4318-945e-86fd2947946a service swift] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:35.736762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:35.737934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:35.738943 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:35.739466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] PATH_INFO: `/v3/projects/afb9becaa4404cf885f80466582421d7` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:35.741367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afb9becaa4404cf885f80466582421d7)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:35.744370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:35.844271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:35.846874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:35.848451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:35.854831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afb9becaa4404cf885f80466582421d7', 'request_id': 'req-c6f7501f-abb6-46f8-903c-9e22074d3b37'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:35.873781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1182/2338] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:35.877074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afb9becaa4404cf885f80466582421d7', 'request_id': 'req-c6f7501f-abb6-46f8-903c-9e22074d3b37'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:35.887719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afb9becaa4404cf885f80466582421d7', 'request_id': 'req-c6f7501f-abb6-46f8-903c-9e22074d3b37'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:35.904034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] The token cache is being invalidate because project afb9becaa4404cf885f80466582421d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:35.913413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:35.918382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:35.920841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1157/2339] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:35 2022] DELETE /identity/v3/projects/afb9becaa4404cf885f80466582421d7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:35.924986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:35.942615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6ccf1-33bf-4c4b-a277-d8bb81b13bf9 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:36.050193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:36.051950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.052445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.052985 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] PATH_INFO: `/v3/users/a9de35ce67fe43f398a58429811d3905` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:36.055342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9de35ce67fe43f398a58429811d3905)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:36.058269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:36.136424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9de35ce67fe43f398a58429811d3905'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.153600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9de35ce67fe43f398a58429811d3905'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.168214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9de35ce67fe43f398a58429811d3905'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.168883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9de35ce67fe43f398a58429811d3905'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.169782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-befc844e-e883-40c0-a400-ca5d21e43c4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9de35ce67fe43f398a58429811d3905'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.208400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1183/2340] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:35 2022] DELETE /identity/v3/users/a9de35ce67fe43f398a58429811d3905 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:36.574707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:36.576180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:36.606531 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7501f-abb6-46f8-903c-9e22074d3b37 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:36.690322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:36.691868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.692259 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.692640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] PATH_INFO: `/v3/projects/fa646cb5798349758ede8814c56129cc` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:36.694150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa646cb5798349758ede8814c56129cc)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:36.696313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:36.741589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:36.766230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa646cb5798349758ede8814c56129cc', 'request_id': 'req-129ebd93-375b-4fe7-ac59-83998d77fdba'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.773241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa646cb5798349758ede8814c56129cc', 'request_id': 'req-129ebd93-375b-4fe7-ac59-83998d77fdba'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.782379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa646cb5798349758ede8814c56129cc', 'request_id': 'req-129ebd93-375b-4fe7-ac59-83998d77fdba'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.800115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] The token cache is being invalidate because project fa646cb5798349758ede8814c56129cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:36.800391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.803145 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-129ebd93-375b-4fe7-ac59-83998d77fdba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:36.808267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1158/2341] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:36 2022] DELETE /identity/v3/projects/fa646cb5798349758ede8814c56129cc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:36.814130 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.814770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.815247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:36.844793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:36.846937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:36.962297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1184/2342] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:36.971632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:36.971632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:36.971632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.009898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:37.011421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.026608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.027763 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1159/2343] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:37.041905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2319c731-de4d-4601-b336-a66f8033dfa6 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.055236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.056366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.057052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.057537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.061420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:37.062404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.067223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.090481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1185/2344] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:37.096070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.111093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfad495-41e7-4db3-9c78-5b6195ea1ece None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.120885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.124114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.124787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.125808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.126805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.131353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.180429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1160/2345] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/projects => generated 393 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:37.189351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.198656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:37.211097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03952faf-8ba0-444f-9b53-6408b951be7b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.274790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.275863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.276219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.276571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.277337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.279610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.309184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1186/2346] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:37.314757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.325690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0eeefb-e8c1-4d7f-9c40-4ebe56b65de8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.330501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41880-eae9-4b14-b777-c8c215204044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.331176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41880-eae9-4b14-b777-c8c215204044 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.331495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41880-eae9-4b14-b777-c8c215204044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.331822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41880-eae9-4b14-b777-c8c215204044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.332773 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41880-eae9-4b14-b777-c8c215204044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.334764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41880-eae9-4b14-b777-c8c215204044 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.350980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.355340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1161/2347] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:37.363139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2f998-bcc2-4ea7-841c-c7aee951c8aa admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.371572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.372413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.372770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.373119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.374024 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.375813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.377193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.377500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.377788 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.399620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1187/2348] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:37.401073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.404481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:37.406074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.416194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32789c-9ce0-431c-8ffb-d58df4fed80b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.423127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79976fd-67b7-4547-9777-b324b17aea4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.424368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79976fd-67b7-4547-9777-b324b17aea4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.424868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79976fd-67b7-4547-9777-b324b17aea4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.425242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79976fd-67b7-4547-9777-b324b17aea4f admin admin] PATH_INFO: `/v3/projects/3d98b8e1f53a436587df393218baff80/users/7ba7fca5750143a3afe1873b1c62494b/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.446970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79976fd-67b7-4547-9777-b324b17aea4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d98b8e1f53a436587df393218baff80, user_id=7ba7fca5750143a3afe1873b1c62494b, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.450777 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79976fd-67b7-4547-9777-b324b17aea4f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.474498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1188/2349] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:37 2022] PUT /identity/v3/projects/3d98b8e1f53a436587df393218baff80/users/7ba7fca5750143a3afe1873b1c62494b/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:37.478116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.478573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.479028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.483500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1162/2350] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:37.505269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.507093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.509154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] MFA Rules not processed for user `7ba7fca5750143a3afe1873b1c62494b`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:37.510246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.607791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71882c57-fb4c-453b-a450-96434b9cad9d None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.609785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1189/2351] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:37.614459 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.627970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:37.672785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:37.678596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.680032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.680391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.680833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.691792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c17051d-b626-4554-80e7-779ee463aedf None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.696725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.697524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.697986 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.698379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.699185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.701203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.729477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1190/2352] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:37.733159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.749185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.751025 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.759887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1163/2353] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:37.805028 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b814980-7e9b-46fb-813a-ce696be45d40 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.810558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.811422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.811775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.812118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.812949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.815103 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.844709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1191/2354] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:37 2022] POST /identity/v3/users => generated 303 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:37.849519 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.864734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf04a84-6feb-43ba-9f0c-b60eb33f37a2 service swift] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.876750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.877744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.878282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.879634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.880623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.887895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.890424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:37.898375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.911343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc16a9-b104-4ef3-8177-b7e169136c5c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.918494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.919728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.920174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.920597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] PATH_INFO: `/v3/users/19cee9d773b541e186dfe7b90aa73e05` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:37.922054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] RBAC: Authorizing `identity:delete_user(user_id=19cee9d773b541e186dfe7b90aa73e05)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:37.925273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:37.933079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1164/2355] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:37.934246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:37.964426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092fb3d-2cb7-4af8-a8cc-b5bccb62ad7a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:37.978712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:37.981736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:37.984202 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:37.984780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] PATH_INFO: `/v3/projects/78958d971b504943bc2d8fc7b72b2a1d/users/62ea86c69a8b43afaa7457e3a22352f4/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.033021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19cee9d773b541e186dfe7b90aa73e05'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:38.056149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78958d971b504943bc2d8fc7b72b2a1d, user_id=62ea86c69a8b43afaa7457e3a22352f4, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:38.065163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:38.068559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19cee9d773b541e186dfe7b90aa73e05'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:38.092652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19cee9d773b541e186dfe7b90aa73e05'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:38.094581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19cee9d773b541e186dfe7b90aa73e05'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:38.099616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44d57c1a-7200-499a-8eaf-3f52ec09647d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19cee9d773b541e186dfe7b90aa73e05'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:38.103277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1165/2356] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:37 2022] PUT /identity/v3/projects/78958d971b504943bc2d8fc7b72b2a1d/users/62ea86c69a8b43afaa7457e3a22352f4/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:38.108901 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:38.113093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:38.144967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1192/2357] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:37 2022] DELETE /identity/v3/users/19cee9d773b541e186dfe7b90aa73e05 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:38.149690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.150229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.150709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.201621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:38.203912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:38.284565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa8369-ccd3-4edc-a9bd-a176aad96c42 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:38.297333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:38.359879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:38.407972 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1193/2358] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:38.418029 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:38.421374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:38.426152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:38.426838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.427133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.429128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.429905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:38.435058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:38.462301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1166/2359] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:38 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:38.463393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:38.487861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d607b3c-e418-432d-809a-d179c3f7a55a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:38.501187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400c1680-2301-484b-a0e0-b4b1ba92d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:38.501775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c1680-2301-484b-a0e0-b4b1ba92d705 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.502057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c1680-2301-484b-a0e0-b4b1ba92d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.502362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400c1680-2301-484b-a0e0-b4b1ba92d705 admin admin] PATH_INFO: `/v3/projects/78958d971b504943bc2d8fc7b72b2a1d/users/62ea86c69a8b43afaa7457e3a22352f4/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.508400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c1680-2301-484b-a0e0-b4b1ba92d705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78958d971b504943bc2d8fc7b72b2a1d, user_id=62ea86c69a8b43afaa7457e3a22352f4, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:38.514592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400c1680-2301-484b-a0e0-b4b1ba92d705 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:38.541625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1167/2360] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:38 2022] PUT /identity/v3/projects/78958d971b504943bc2d8fc7b72b2a1d/users/62ea86c69a8b43afaa7457e3a22352f4/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:38.555117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.555461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.555740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:38.624476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] MFA Rules not processed for user `62ea86c69a8b43afaa7457e3a22352f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:38.625845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:38.685047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6519da6-0b5d-4540-910f-90af2ab434f5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:38.778024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1168/2361] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:43:38.793480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:38.794384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:38.797144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:38.797587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.038485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:39.040711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:39.054641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1194/2362] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 638 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:39.145413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:39.147639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.294696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee9cd2a-bb52-4fae-89f1-b180eda94ee9 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:39.307270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:39.307986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.308274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.308556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.328872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:39.330413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:39.383638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1169/2363] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:39 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:39.393774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:39.395895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.424594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:39.427133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.435767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a640b-744b-4146-9e44-fd4f69c20a42 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:39.452782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:39.453730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.454054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.454437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] PATH_INFO: `/v3/projects/46c24775038b45598100e637f569a117` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.462626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46c24775038b45598100e637f569a117)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:39.472830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e882eb-e007-40e5-9fce-8e2d0747c589 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:39.472969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:39.487185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:39.488711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.489152 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.489408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] PATH_INFO: `/v3/users/9e304a7a988747b68265df63e05ec9a5` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.493611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e304a7a988747b68265df63e05ec9a5)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:39.498816 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:39.569213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:39.608505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46c24775038b45598100e637f569a117', 'request_id': 'req-bc0e97f2-8658-4ef4-9198-4716b82e3510'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.611547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:39.620645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e304a7a988747b68265df63e05ec9a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.629748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46c24775038b45598100e637f569a117', 'request_id': 'req-bc0e97f2-8658-4ef4-9198-4716b82e3510'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.648911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46c24775038b45598100e637f569a117', 'request_id': 'req-bc0e97f2-8658-4ef4-9198-4716b82e3510'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.654002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e304a7a988747b68265df63e05ec9a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.690244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] The token cache is being invalidate because project 46c24775038b45598100e637f569a117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:39.690905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.695154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.697188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1195/2364] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:39 2022] DELETE /identity/v3/projects/46c24775038b45598100e637f569a117 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:39.699569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:39.701514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.715408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e304a7a988747b68265df63e05ec9a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.726224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e304a7a988747b68265df63e05ec9a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.731892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49026046-cfc4-48a0-9861-6e77296e232f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e304a7a988747b68265df63e05ec9a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:39.740547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1170/2365] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:39 2022] DELETE /identity/v3/users/9e304a7a988747b68265df63e05ec9a5 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:39.744383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a1afd-193c-4d23-a00b-296ea0ffb789 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.744626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a1afd-193c-4d23-a00b-296ea0ffb789 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.744863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7a1afd-193c-4d23-a00b-296ea0ffb789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.785456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-aa7a1afd-193c-4d23-a00b-296ea0ffb789 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:39.786689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-aa7a1afd-193c-4d23-a00b-296ea0ffb789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.871585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e97f2-8658-4ef4-9198-4716b82e3510 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:39.918946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1171/2366] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:39.921893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737114d9-c8fc-464c-8941-1662533cc42d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.922187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737114d9-c8fc-464c-8941-1662533cc42d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.922467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737114d9-c8fc-464c-8941-1662533cc42d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:39.967020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-737114d9-c8fc-464c-8941-1662533cc42d None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:39.968288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-737114d9-c8fc-464c-8941-1662533cc42d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:39.987898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1172/2367] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:39.991549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:39.991839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:39.992095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.012374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.013101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.013385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.013665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.024755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:40.040516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:40.043972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:40.064946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1173/2368] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:40.065438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.068761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:40.184599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.186296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.214582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1196/2369] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:40.215389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.243856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8b704-87d4-4a0d-b0f0-96dc1bef7be1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:40.262910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f6bd8f-7a9d-4084-8e35-8813ac46cb2b service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:40.264376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.265338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.265664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.265991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.282284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.282969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.283354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.283637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.284758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:40.294691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.296877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.298468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.306001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.350689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1197/2370] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:43:40.351011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1174/2371] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:40.351798 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.370414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af929bd-d3ef-4e27-bb14-f05f05c7f529 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:40.385289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.386021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.386392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.386660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.387632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.392973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.449117 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:40.464714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1198/2372] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:40 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:40.468948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.483715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9d13e-5ffb-4e3d-9366-0d182e724f7f service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:40.723759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:40.751011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.751978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.752443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.753233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.755443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.757803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.786171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1175/2373] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:40 2022] POST /identity/v3/users => generated 287 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:40.790735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.823510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a239977-84b9-4659-8790-cd6b739f3c23 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:40.836081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.837348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.837828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.838259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.839493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.845629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.896585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1199/2374] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:40.906787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.930796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96fe20-f9da-4db0-a159-fe29f1fccb6a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:40.945677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:40.946806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:40.947439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:40.947779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:40.948907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:40.950870 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:40.964298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:40.973087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:40.974867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1176/2375] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:40.979421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:41.007919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc26df9-a111-4b17-b19f-aded4cd15ff4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:41.011963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca6a93-d05b-49c3-9ac0-c7974d05b30f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:41.035487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a7aab0-5f39-4887-bea8-eab7db510c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:41.036403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7aab0-5f39-4887-bea8-eab7db510c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:41.036740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7aab0-5f39-4887-bea8-eab7db510c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:41.036948 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a7aab0-5f39-4887-bea8-eab7db510c8f admin admin] PATH_INFO: `/v3/projects/d567a9e7ba344ca4ac65b276b57995d1/users/7eb223c95f794ebd8d34f3a8627dfd89/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:41.056683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:41.057579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:41.057862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:41.058137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] PATH_INFO: `/v3/projects/0e05f049758043cb9c0c1095bc4e0b95` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:41.059917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e05f049758043cb9c0c1095bc4e0b95)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:41.061845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:41.085274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7aab0-5f39-4887-bea8-eab7db510c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d567a9e7ba344ca4ac65b276b57995d1, user_id=7eb223c95f794ebd8d34f3a8627dfd89, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:41.088653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a7aab0-5f39-4887-bea8-eab7db510c8f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:41.123613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1177/2376] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:40 2022] PUT /identity/v3/projects/d567a9e7ba344ca4ac65b276b57995d1/users/7eb223c95f794ebd8d34f3a8627dfd89/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:41.136600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:41.136923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:41.137226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:41.165610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:41.168163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e05f049758043cb9c0c1095bc4e0b95', 'request_id': 'req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.171149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:41.175204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e05f049758043cb9c0c1095bc4e0b95', 'request_id': 'req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.192091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e05f049758043cb9c0c1095bc4e0b95', 'request_id': 'req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.206419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] The token cache is being invalidate because project 0e05f049758043cb9c0c1095bc4e0b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:41.207364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.207931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.209596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1200/2377] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:40 2022] DELETE /identity/v3/projects/0e05f049758043cb9c0c1095bc4e0b95 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:41.220243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:41.225809 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:41.259429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1178/2378] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:41.267430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:41.268844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:41.303710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f22eb9-2d5f-4d49-aa95-84f423f0e3cb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:41.309451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:41.311283 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:41.311698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:41.311981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] PATH_INFO: `/v3/users/41d98595894a443fa0a9f358af0b78aa` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:41.313629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d98595894a443fa0a9f358af0b78aa)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:41.315968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:41.330246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eafb896-ff90-4a25-a596-abed69ca97fa None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:41.334723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:41.335480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:41.335872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:41.336102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:41.346376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:41.348085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:41.358052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1179/2379] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:43:41.370585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d98595894a443fa0a9f358af0b78aa'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.377815 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d98595894a443fa0a9f358af0b78aa'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.385652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d98595894a443fa0a9f358af0b78aa'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.386288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d98595894a443fa0a9f358af0b78aa'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.386915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d98595894a443fa0a9f358af0b78aa'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:41.393990 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:41.407057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1201/2380] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:41 2022] DELETE /identity/v3/users/41d98595894a443fa0a9f358af0b78aa => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:42.430676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:42.441829 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b256e2d-9816-40e0-9020-f116b9fc7e2a service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:42.514727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:42.535584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:42.536573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:42.536861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:42.537261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] PATH_INFO: `/v3/projects/273ca832d6f1464cbe2f428b06e28019` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:42.538499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=273ca832d6f1464cbe2f428b06e28019)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:42.540526 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:42.591014 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '273ca832d6f1464cbe2f428b06e28019', 'request_id': 'req-cc71b686-a938-4991-b0a7-1eb270be6f1d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.601349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '273ca832d6f1464cbe2f428b06e28019', 'request_id': 'req-cc71b686-a938-4991-b0a7-1eb270be6f1d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.620478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '273ca832d6f1464cbe2f428b06e28019', 'request_id': 'req-cc71b686-a938-4991-b0a7-1eb270be6f1d'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.653924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] The token cache is being invalidate because project 273ca832d6f1464cbe2f428b06e28019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:42.654139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.654905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.656854 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1180/2381] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:42 2022] DELETE /identity/v3/projects/273ca832d6f1464cbe2f428b06e28019 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:42.660550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:42.662552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:42.680828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d6684-3247-4688-815f-f391c1202dbb admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:42.793031 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:42.793938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:42.794217 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:42.794487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] PATH_INFO: `/v3/users/d3aff2964db541a18e49f584f66c6795` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:42.800667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3aff2964db541a18e49f584f66c6795)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:42.806895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:42.869696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3aff2964db541a18e49f584f66c6795'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.877596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3aff2964db541a18e49f584f66c6795'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.888251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3aff2964db541a18e49f584f66c6795'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.888923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3aff2964db541a18e49f584f66c6795'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.889737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-44e7fc14-4e9a-4468-8ed7-17a17a9efe17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3aff2964db541a18e49f584f66c6795'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:42.927018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1202/2382] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:43:42 2022] DELETE /identity/v3/users/d3aff2964db541a18e49f584f66c6795 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:43.408170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:43.408795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:43.437101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc71b686-a938-4991-b0a7-1eb270be6f1d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:43.599419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:43.601491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:43.602550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:43.603696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] PATH_INFO: `/v3/projects/4d40bee618b440ce93282bf5a0c71c2f` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:43.607020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d40bee618b440ce93282bf5a0c71c2f)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:43.614400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:43.691490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d40bee618b440ce93282bf5a0c71c2f', 'request_id': 'req-0c01f36a-cade-45e7-9a2d-88819341cc7a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:43.700396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d40bee618b440ce93282bf5a0c71c2f', 'request_id': 'req-0c01f36a-cade-45e7-9a2d-88819341cc7a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:43.715421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d40bee618b440ce93282bf5a0c71c2f', 'request_id': 'req-0c01f36a-cade-45e7-9a2d-88819341cc7a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:43.738237 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] The token cache is being invalidate because project 4d40bee618b440ce93282bf5a0c71c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:43:43.738632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:43.739552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0c01f36a-cade-45e7-9a2d-88819341cc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:43:43.741608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1181/2383] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:43:43 2022] DELETE /identity/v3/projects/4d40bee618b440ce93282bf5a0c71c2f => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:43.748882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:43.749851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:43.750315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:43.817102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:43.820237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:43.987279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1203/2384] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:43.993353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:43.994338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:43.994611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.015619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:44.032162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:44.044758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:44.056191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1182/2385] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:44.060146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.073999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d6315-37cd-49fb-bb93-dc17a235951a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.079452 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.080392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.094240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.101701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.101701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:43:44.101701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.102530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.115489 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:44.129227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1204/2386] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:43:44.132650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.149782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af324d-5f39-4c39-a6d1-f4438eecac97 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.156471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.157315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.157610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.157886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.158632 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.160666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.193099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1183/2387] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:44 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:44.197036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.211306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da30ef6b-c547-4456-b418-f9b5df58b686 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.290601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.291369 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.291657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.292609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.292905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.295055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.329824 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1205/2388] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:44 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:44.330569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.341978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fae9052-7188-4fa1-bf84-03cce614cf70 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.349952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.350647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.350936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.351239 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.351987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.353651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.373831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1184/2389] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:44.377851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.391163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98175dc-1406-460d-b364-63b74c4be5e7 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.397948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf6733-5ece-40b0-8326-42250b36f998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.399096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf6733-5ece-40b0-8326-42250b36f998 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.399592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf6733-5ece-40b0-8326-42250b36f998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.400225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf6733-5ece-40b0-8326-42250b36f998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.401132 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf6733-5ece-40b0-8326-42250b36f998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.402931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf6733-5ece-40b0-8326-42250b36f998 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.415803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1206/2390] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:44.420027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.431979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15191838-af83-49eb-b6f2-d1761b1fa2c0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.436793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.437686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.438282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.438489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] PATH_INFO: `/v3/projects/6a5c5193e1994a57a250068bb347b259/users/3b5932082fd34372ad6c205e43538166/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.469545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a5c5193e1994a57a250068bb347b259, user_id=3b5932082fd34372ad6c205e43538166, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.472407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.502077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1185/2391] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:44 2022] PUT /identity/v3/projects/6a5c5193e1994a57a250068bb347b259/users/3b5932082fd34372ad6c205e43538166/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:44.515322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ecf15-c93e-4086-b7e2-0653d7eb1f64 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.515646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ecf15-c93e-4086-b7e2-0653d7eb1f64 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.515905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ecf15-c93e-4086-b7e2-0653d7eb1f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.568246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3d0ecf15-c93e-4086-b7e2-0653d7eb1f64 None None] MFA Rules not processed for user `3b5932082fd34372ad6c205e43538166`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:44.569854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3d0ecf15-c93e-4086-b7e2-0653d7eb1f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:44.685007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1207/2392] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:44.692669 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:44.694623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:44.777373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aea78e4-2666-422f-83da-a400ec779740 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:44.781628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:44.782499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:44.782795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:44.783560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:44.794377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:44.796287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:44.807690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1186/2393] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:44 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:47.202162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.202670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.202787 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.235568 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] MFA Rules not processed for user `7eb223c95f794ebd8d34f3a8627dfd89`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:47.236789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:47.300599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:47.319525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1208/2394] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:47.319757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:47.321462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:47.398535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8086e4-cb2f-48e8-9006-3148e868355a service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:47.403481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:47.404146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.404434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.404853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.405564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:47.411625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:47.443678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1187/2395] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:43:47 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:43:47.447724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:47.458781 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c5369a-012b-4141-98d3-d1ebea0a5b6f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:47.521549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:47.522220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.522501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.522776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.523646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:47.525547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:47.546326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1209/2396] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:43:47 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:43:47.550313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:47.561049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911be769-d943-4add-b2bb-98325db27e99 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:47.565864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:47.566554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.566866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.567135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.567970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:47.569575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:47.582524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1188/2397] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:43:47.588180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:47.618789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92e1fa2-1edc-4774-aa35-97f49ff9fd4f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:47.640998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:47.641799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.642094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.642372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] PATH_INFO: `/v3/projects/64e08dd6d4d14f5981d81fad90d7bd34/users/81ff325bd75b4e478efcd9beda30eb31/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.667691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e08dd6d4d14f5981d81fad90d7bd34, user_id=81ff325bd75b4e478efcd9beda30eb31, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:47.671116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:47.693724 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1210/2398] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:47 2022] PUT /identity/v3/projects/64e08dd6d4d14f5981d81fad90d7bd34/users/81ff325bd75b4e478efcd9beda30eb31/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:43:47.697827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:47.708999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16396235-6575-4741-b6f3-b139766f3d68 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:47.728539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:47.775525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae9650-4b8e-4b30-a6c5-89e89b1b0f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:47.776310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae9650-4b8e-4b30-a6c5-89e89b1b0f27 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.776656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae9650-4b8e-4b30-a6c5-89e89b1b0f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.776997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae9650-4b8e-4b30-a6c5-89e89b1b0f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.777798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae9650-4b8e-4b30-a6c5-89e89b1b0f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:47.779727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae9650-4b8e-4b30-a6c5-89e89b1b0f27 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:47.790436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1189/2399] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:43:47.795069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:47.805773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94be2b88-9966-4fa2-baec-e4215098491e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:47.810429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:47.811386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:47.811669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:47.811944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] PATH_INFO: `/v3/projects/64e08dd6d4d14f5981d81fad90d7bd34/users/81ff325bd75b4e478efcd9beda30eb31/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:47.813906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64e08dd6d4d14f5981d81fad90d7bd34, user_id=81ff325bd75b4e478efcd9beda30eb31, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:47.817131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:47.828760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1211/2400] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:43:47 2022] PUT /identity/v3/projects/64e08dd6d4d14f5981d81fad90d7bd34/users/81ff325bd75b4e478efcd9beda30eb31/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:43:53.479168 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78f19-7f61-4643-961c-380e8e2f9c3d None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:53.479779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78f19-7f61-4643-961c-380e8e2f9c3d None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:53.479779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78f19-7f61-4643-961c-380e8e2f9c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:53.552041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-3cc78f19-7f61-4643-961c-380e8e2f9c3d None None] MFA Rules not processed for user `81ff325bd75b4e478efcd9beda30eb31`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:53.555232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3cc78f19-7f61-4643-961c-380e8e2f9c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:53.717371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1190/2401] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:53.858571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:53.860535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:53.974633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbeba6d7-7d4f-4191-818d-3945590cd675 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:53.982655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:53.983594 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:53.983884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:53.984235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:54.038274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:43:54.058902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:54.061460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:54.073013 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1212/2402] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:43:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:43:54.128328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232490bd-9551-4cc5-a1e8-b94ef306978b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:54.129214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232490bd-9551-4cc5-a1e8-b94ef306978b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:54.129560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232490bd-9551-4cc5-a1e8-b94ef306978b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:54.161535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-232490bd-9551-4cc5-a1e8-b94ef306978b None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:54.163116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-232490bd-9551-4cc5-a1e8-b94ef306978b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:54.247565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1191/2403] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:43:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:43:54.270227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:43:54.272730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:54.389679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd4e2f7-94c9-4087-a012-e11b23c65d35 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:43:54.398463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:43:54.403324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:54.403324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:54.403324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:54.433158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:43:54.435936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:43:54.455833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1213/2404] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:43:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:43:56.459754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bf654-16f9-44d2-8519-1e81fa022ff1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:43:56.460354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bf654-16f9-44d2-8519-1e81fa022ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:43:56.460354 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bf654-16f9-44d2-8519-1e81fa022ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:43:56.484569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-0f7bf654-16f9-44d2-8519-1e81fa022ff1 None None] MFA Rules not processed for user `7eb223c95f794ebd8d34f3a8627dfd89`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:43:56.485817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0f7bf654-16f9-44d2-8519-1e81fa022ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:43:56.494012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1192/2405] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:43:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:02.699023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:02.707982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:02.796649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab73e2-6ed8-4db5-a764-3eec053649ef service swift] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:02.803079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:02.804109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.804397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.804672 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] PATH_INFO: `/v3/users/7ba7fca5750143a3afe1873b1c62494b` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:02.805864 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba7fca5750143a3afe1873b1c62494b)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:02.808233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:02.924109 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba7fca5750143a3afe1873b1c62494b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.932778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba7fca5750143a3afe1873b1c62494b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.944143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba7fca5750143a3afe1873b1c62494b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.947842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba7fca5750143a3afe1873b1c62494b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.948751 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba7fca5750143a3afe1873b1c62494b'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:02.961609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:02.980557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1214/2406] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:02 2022] DELETE /identity/v3/users/7ba7fca5750143a3afe1873b1c62494b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:02.984717 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:02.985329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:02.985706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.008497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:03.010304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:03.056655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:03.099428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1193/2407] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:03.106211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.108144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:03.195399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b939e8d-ea30-4053-b927-9ae28b93687e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.200245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.201154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.201569 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.201843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.213136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.214856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.226793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1215/2408] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:44:03.546868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.548207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:03.569071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c655f-a788-41f1-9539-a0aeffdf98ff None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.577548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.578510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.578862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.579156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] PATH_INFO: `/v3/projects/3d98b8e1f53a436587df393218baff80` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.580407 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d98b8e1f53a436587df393218baff80)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.583171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.639234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d98b8e1f53a436587df393218baff80', 'request_id': 'req-561f7d06-5654-4a38-9328-3a315484a5c5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.649003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d98b8e1f53a436587df393218baff80', 'request_id': 'req-561f7d06-5654-4a38-9328-3a315484a5c5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.657594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d98b8e1f53a436587df393218baff80', 'request_id': 'req-561f7d06-5654-4a38-9328-3a315484a5c5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.669945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] The token cache is being invalidate because project 3d98b8e1f53a436587df393218baff80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:03.670249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.670925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.672911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1194/2409] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:03 2022] DELETE /identity/v3/projects/3d98b8e1f53a436587df393218baff80 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:03.681455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:03.701775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0b0f24-9382-4ebb-aa40-569557c15acd service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:03.799331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:03.799331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:03.799331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:03.799331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] PATH_INFO: `/v3/users/62ea86c69a8b43afaa7457e3a22352f4` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:03.800251 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62ea86c69a8b43afaa7457e3a22352f4)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:03.803180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:03.859395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62ea86c69a8b43afaa7457e3a22352f4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.870400 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62ea86c69a8b43afaa7457e3a22352f4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.880690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62ea86c69a8b43afaa7457e3a22352f4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.881592 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62ea86c69a8b43afaa7457e3a22352f4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.882237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-300e3a75-63f9-42d5-86a6-873e7d65e874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62ea86c69a8b43afaa7457e3a22352f4'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:03.909444 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1216/2410] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:03 2022] DELETE /identity/v3/users/62ea86c69a8b43afaa7457e3a22352f4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:04.325714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:04.327409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:04.466308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f7d06-5654-4a38-9328-3a315484a5c5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:04.538731 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:04.539682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:04.539974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:04.540251 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] PATH_INFO: `/v3/projects/78958d971b504943bc2d8fc7b72b2a1d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:04.541454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78958d971b504943bc2d8fc7b72b2a1d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:04.543920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:04.546250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:04.641525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78958d971b504943bc2d8fc7b72b2a1d', 'request_id': 'req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:04.654329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78958d971b504943bc2d8fc7b72b2a1d', 'request_id': 'req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:04.663523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78958d971b504943bc2d8fc7b72b2a1d', 'request_id': 'req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:04.679684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] The token cache is being invalidate because project 78958d971b504943bc2d8fc7b72b2a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:04.680271 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:04.681577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-a3a7dccd-bcc8-4f42-9aff-87ce05e1beb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:04.683226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1195/2411] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:04 2022] DELETE /identity/v3/projects/78958d971b504943bc2d8fc7b72b2a1d => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:04.688828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:04.689270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:04.689564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:04.735679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:04.819318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:04.821850 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:04.911930 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1217/2412] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:04.925281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:04.925820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:04.926540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:04.969679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:04.969679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:05.115895 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.120083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1196/2413] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:05.182171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977efb0-98d4-44f4-84a6-b0b2fb7fcef2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.197968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.198971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.199359 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.199748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.200668 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:05.201089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.203588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:05.226507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.232897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1218/2414] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:44:05.240663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0391290c-9c9e-4c71-8c2b-2ec2dcd9e8c6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.247039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.248085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.248374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.248654 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.249666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.252073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:05.303595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1197/2415] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:05 2022] POST /identity/v3/projects => generated 365 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:05.308163 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.322330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7d7fc-9c37-4324-a8bd-e7bfaff3bde5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.431447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.432411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.432819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.433169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.434287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.436804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:05.505060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1219/2416] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:05 2022] POST /identity/v3/users => generated 283 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:05.507580 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.526513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87f3cc1-a115-4166-8768-01b9957cfc75 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.532864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.533857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.534262 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.535595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.536692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.540900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:05.559645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1198/2417] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:44:05.563343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.584870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fccb9e8-cd1e-4733-9d68-a45d36a91c6b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.603597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0391bf0-fefc-458d-96ef-dd5da2b90fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.603597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0391bf0-fefc-458d-96ef-dd5da2b90fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.603597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0391bf0-fefc-458d-96ef-dd5da2b90fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.603597 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0391bf0-fefc-458d-96ef-dd5da2b90fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.604073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0391bf0-fefc-458d-96ef-dd5da2b90fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.604073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0391bf0-fefc-458d-96ef-dd5da2b90fbb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:05.616605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1220/2418] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:05.620611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.634386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03b3dc-41dc-4543-896a-9600435458b4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.641090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.642107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.642495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.643404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] PATH_INFO: `/v3/projects/97ddf8ca1e0d41a787c8ee0deefc10b3/users/72a04bd820174befb7448a517f7bf326/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.670051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ddf8ca1e0d41a787c8ee0deefc10b3, user_id=72a04bd820174befb7448a517f7bf326, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.673400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:05.696582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1199/2419] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:05 2022] PUT /identity/v3/projects/97ddf8ca1e0d41a787c8ee0deefc10b3/users/72a04bd820174befb7448a517f7bf326/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:05.706446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.707096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.708173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.751340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] MFA Rules not processed for user `72a04bd820174befb7448a517f7bf326`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:05.753327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:05.811609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:05.844588 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1221/2420] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:05 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:05.845914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:05.928245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3c22d-bed3-4840-8573-474c5c9e47eb admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:05.941252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:05.942574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:05.942966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:05.943401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:05.944404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:05.946863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.059532 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1200/2421] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:05 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:06.063074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.074898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15dcd66-c1f0-4bfc-b81b-89582d653b93 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.137427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.138155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.138460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.138745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.139565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:06.141526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.183344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1222/2422] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:06 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:06.184349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.195471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d2355-b149-47ab-8f0b-4c91bb20c875 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.199963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.200620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.200905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.201182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.201908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:06.203554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.214475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1201/2423] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:06.219462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.229923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00cc9e8-2ff3-4b4b-b3fe-10c551bd2423 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.234649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.235575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.235854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.236129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] PATH_INFO: `/v3/projects/4fac5a3aa3ad41d4a2d314f435ba947a/users/dcbf3275849e4c78842e1e50c4f7a792/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.266134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fac5a3aa3ad41d4a2d314f435ba947a, user_id=dcbf3275849e4c78842e1e50c4f7a792, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:06.269135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.286448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1223/2424] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:06 2022] PUT /identity/v3/projects/4fac5a3aa3ad41d4a2d314f435ba947a/users/dcbf3275849e4c78842e1e50c4f7a792/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:44:06.290980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.296909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:06.304995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9411d5-e385-4066-b760-5c2d80d4ae49 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.377778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285db7f1-841a-4373-9d74-22c7cf02a3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.378644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db7f1-841a-4373-9d74-22c7cf02a3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.379160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db7f1-841a-4373-9d74-22c7cf02a3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.379696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285db7f1-841a-4373-9d74-22c7cf02a3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.380453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db7f1-841a-4373-9d74-22c7cf02a3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:06.382190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285db7f1-841a-4373-9d74-22c7cf02a3c1 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.393805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1202/2425] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:06 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:06.397981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.417992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516c46d-3291-4cd8-bd37-86c5b0650188 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.424922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.426022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.426426 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.426619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] PATH_INFO: `/v3/projects/4fac5a3aa3ad41d4a2d314f435ba947a/users/dcbf3275849e4c78842e1e50c4f7a792/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.429110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fac5a3aa3ad41d4a2d314f435ba947a, user_id=dcbf3275849e4c78842e1e50c4f7a792, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:06.432204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.446140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1224/2426] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:06 2022] PUT /identity/v3/projects/4fac5a3aa3ad41d4a2d314f435ba947a/users/dcbf3275849e4c78842e1e50c4f7a792/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:06.457462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.458082 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.458615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.489934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] MFA Rules not processed for user `dcbf3275849e4c78842e1e50c4f7a792`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:06.491375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:06.574102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1203/2427] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:06 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:06.587040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.588804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:06.727530 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731d8020-3604-4d99-be1a-86ca19b44c93 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.733086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.734183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.734487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.735090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:06.760891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:06.762833 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:06.791072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1225/2428] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:06 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:06.898292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:06.912467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780d0c7-ac93-4617-9ad8-e518691f16be None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:06.934952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e19795-da73-48fb-8381-1d8153b77655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:06.935673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e19795-da73-48fb-8381-1d8153b77655 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:06.935959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e19795-da73-48fb-8381-1d8153b77655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:06.936236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e19795-da73-48fb-8381-1d8153b77655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:07.025664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e19795-da73-48fb-8381-1d8153b77655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:07.027113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e19795-da73-48fb-8381-1d8153b77655 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:07.037794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1204/2429] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:06 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:11.268252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:11.269712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:11.318762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:11.369504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26d69d-6152-4fdb-b79f-2546ab499ccd service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:11.374564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:11.375472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:11.375757 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:11.376032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] PATH_INFO: `/v3/users/ce70428ff29d416c9bcfaca85f7cdf7e` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:11.377260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce70428ff29d416c9bcfaca85f7cdf7e)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:11.379541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:11.435898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce70428ff29d416c9bcfaca85f7cdf7e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:11.444632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce70428ff29d416c9bcfaca85f7cdf7e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:11.453372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce70428ff29d416c9bcfaca85f7cdf7e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:11.454035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce70428ff29d416c9bcfaca85f7cdf7e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:11.454659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce70428ff29d416c9bcfaca85f7cdf7e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:11.474224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1226/2430] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:11 2022] DELETE /identity/v3/users/ce70428ff29d416c9bcfaca85f7cdf7e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:11.478797 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:11.479971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:11.479971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:11.498099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:11.499381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:11.520113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:11.574907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1205/2431] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:11.585738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:11.588081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:11.791245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4241716-bb2e-4cd4-b35a-505e617983df admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:11.797734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:11.812428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:11.812869 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:11.823716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:11.837222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:11.844547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:11.872661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1227/2432] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:44:12.574833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:12.576949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:12.601757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f423763-7a67-4a57-b948-a1ae60e6da39 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:12.613852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:12.614861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:12.621777 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:12.623113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] PATH_INFO: `/v3/projects/aebde08df0944391bb35a94e7541f20d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:12.628861 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aebde08df0944391bb35a94e7541f20d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:12.631782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:12.713357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aebde08df0944391bb35a94e7541f20d', 'request_id': 'req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:12.725437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aebde08df0944391bb35a94e7541f20d', 'request_id': 'req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:12.733406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aebde08df0944391bb35a94e7541f20d', 'request_id': 'req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:12.746078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] The token cache is being invalidate because project aebde08df0944391bb35a94e7541f20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:12.746548 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:12.747293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0b3721b7-d5aa-49f4-94d6-bbc3b051eec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:12.751289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1206/2433] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:12 2022] DELETE /identity/v3/projects/aebde08df0944391bb35a94e7541f20d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:14.719748 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:14.722238 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:14.740707 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a47dcc-8b0b-404d-835d-afeff20d09a1 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:14.821094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:14.936447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:14.937570 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:14.937900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:14.938202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] PATH_INFO: `/v3/users/72a04bd820174befb7448a517f7bf326` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:14.940018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a04bd820174befb7448a517f7bf326)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:14.942168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:14.993782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a04bd820174befb7448a517f7bf326'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.000368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a04bd820174befb7448a517f7bf326'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.012868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a04bd820174befb7448a517f7bf326'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.014244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a04bd820174befb7448a517f7bf326'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.015037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a04bd820174befb7448a517f7bf326'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:15.034846 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1228/2434] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:14 2022] DELETE /identity/v3/users/72a04bd820174befb7448a517f7bf326 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:15.039871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:15.040389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:15.040547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:15.059089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:15.060513 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:15.129169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1207/2435] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:15.139936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:15.141516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:15.222364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d019d929-53c7-415d-9698-a19c2e053cff admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:15.226652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:15.227424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:15.227710 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:15.228000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:15.238337 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:15.239957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:15.251344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1229/2436] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:15.796951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:15.798705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:15.823659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4f5e89-a83d-4e28-acd7-efb5e68bb200 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:15.851367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:15.852694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:15.858928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:15.860133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] PATH_INFO: `/v3/projects/97ddf8ca1e0d41a787c8ee0deefc10b3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:15.871600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ddf8ca1e0d41a787c8ee0deefc10b3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:15.879111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:16.047729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:16.072083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ddf8ca1e0d41a787c8ee0deefc10b3', 'request_id': 'req-e8f637ac-106c-424d-b13a-2c4f0b718a28'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.097828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ddf8ca1e0d41a787c8ee0deefc10b3', 'request_id': 'req-e8f637ac-106c-424d-b13a-2c4f0b718a28'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.118662 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ddf8ca1e0d41a787c8ee0deefc10b3', 'request_id': 'req-e8f637ac-106c-424d-b13a-2c4f0b718a28'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.134482 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] The token cache is being invalidate because project 97ddf8ca1e0d41a787c8ee0deefc10b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:16.134832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.135494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.142280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1208/2437] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:15 2022] DELETE /identity/v3/projects/97ddf8ca1e0d41a787c8ee0deefc10b3 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:16.143113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:16.155755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688c8f3e-0272-4c60-8376-4c1c13c2e6ec service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:16.238049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:16.250931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:16.251851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:16.252131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:16.252413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] PATH_INFO: `/v3/users/dcbf3275849e4c78842e1e50c4f7a792` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:16.254297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcbf3275849e4c78842e1e50c4f7a792)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:16.256061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:16.311782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcbf3275849e4c78842e1e50c4f7a792'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.320054 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcbf3275849e4c78842e1e50c4f7a792'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.329756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcbf3275849e4c78842e1e50c4f7a792'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.330408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcbf3275849e4c78842e1e50c4f7a792'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.331040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-fcfbc21f-91e8-48f9-8ff0-ccf056f34ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcbf3275849e4c78842e1e50c4f7a792'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.350862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1230/2438] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:16 2022] DELETE /identity/v3/users/dcbf3275849e4c78842e1e50c4f7a792 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:44:16.698341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:16.700325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:16.714228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f637ac-106c-424d-b13a-2c4f0b718a28 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:16.788712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:16.791176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:16.791557 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:16.792016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] PATH_INFO: `/v3/projects/4fac5a3aa3ad41d4a2d314f435ba947a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:16.793974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fac5a3aa3ad41d4a2d314f435ba947a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:16.796715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:16.864612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fac5a3aa3ad41d4a2d314f435ba947a', 'request_id': 'req-79b65cb6-a52f-467d-93c5-2a81c8a5665c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.874606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fac5a3aa3ad41d4a2d314f435ba947a', 'request_id': 'req-79b65cb6-a52f-467d-93c5-2a81c8a5665c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.883740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fac5a3aa3ad41d4a2d314f435ba947a', 'request_id': 'req-79b65cb6-a52f-467d-93c5-2a81c8a5665c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.897983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] The token cache is being invalidate because project 4fac5a3aa3ad41d4a2d314f435ba947a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:16.898558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.899245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79b65cb6-a52f-467d-93c5-2a81c8a5665c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:16.901877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1209/2439] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:16 2022] DELETE /identity/v3/projects/4fac5a3aa3ad41d4a2d314f435ba947a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:16.907144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:16.907548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:16.907825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:16.955485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:16.957051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:17.076258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.076867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.077209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.079793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1231/2440] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:17.103233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:17.103233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:17.116316 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1210/2441] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:17.120940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.133363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8e57f7-5bff-48cd-abfa-c0f4c1607154 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.140295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.141492 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.141881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.142905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.144387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:17.145001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.147403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.171368 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1232/2442] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:44:17.178573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.185823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b175b415-7ae3-4bd6-b45b-8d9d66ad671e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.204544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.205618 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.206163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.206495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.207864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.211771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.241692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1211/2443] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:17 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:17.246418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.264590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdc3d9-ad61-4e80-9f9d-88367bb32baf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.348219 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:17.366627 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.367733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.368642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.369159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.370172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.372915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.400749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1233/2444] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:17 2022] POST /identity/v3/users => generated 293 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:17.405934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.418897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c1d6b0-a15c-4c6b-a98b-2877e1232828 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.425847 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.426798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.427359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.427837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.428945 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.431384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.445032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1212/2445] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:17.449267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.462374 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae172db-b182-47eb-89c1-0cb24e15a97e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.468048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52636d0a-8c65-43d5-8b26-be1354e7012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.469320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52636d0a-8c65-43d5-8b26-be1354e7012c admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.469968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52636d0a-8c65-43d5-8b26-be1354e7012c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.470801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52636d0a-8c65-43d5-8b26-be1354e7012c admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.475285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52636d0a-8c65-43d5-8b26-be1354e7012c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.477535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52636d0a-8c65-43d5-8b26-be1354e7012c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.490441 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1234/2446] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:17.499902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.512116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d521cd4-b46f-4328-b5dc-990a4fb3ce2f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.518378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.520154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.520682 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.521176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] PATH_INFO: `/v3/projects/daafa90f8b164d30ae44ba253470c82f/users/17b6e7ffad324e2aaafef691026fe15a/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.543609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=daafa90f8b164d30ae44ba253470c82f, user_id=17b6e7ffad324e2aaafef691026fe15a, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.546855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.550512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:17.572472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1213/2447] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:17 2022] PUT /identity/v3/projects/daafa90f8b164d30ae44ba253470c82f/users/17b6e7ffad324e2aaafef691026fe15a/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:44:17.574470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.574768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.575052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.603645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] MFA Rules not processed for user `17b6e7ffad324e2aaafef691026fe15a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:17.604885 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:17.683067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1235/2448] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:17.696129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.697944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:17.778467 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f64780-ee8e-459d-9679-b7b5aaa613cf admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.782587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287004f-b467-4837-a1f9-b63d03a9efcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.783270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287004f-b467-4837-a1f9-b63d03a9efcb service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.785968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287004f-b467-4837-a1f9-b63d03a9efcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.785968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287004f-b467-4837-a1f9-b63d03a9efcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.794834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287004f-b467-4837-a1f9-b63d03a9efcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.796756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287004f-b467-4837-a1f9-b63d03a9efcb service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:17.809764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1214/2449] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:17.835155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:17.917708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223be90e-54c1-42bf-9750-0df1cf2d2896 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:17.923439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:17.924631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:17.924991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:17.925365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] PATH_INFO: `/v3/users/17b6e7ffad324e2aaafef691026fe15a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:17.926837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17b6e7ffad324e2aaafef691026fe15a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:17.929153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:18.001889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17b6e7ffad324e2aaafef691026fe15a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.011687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17b6e7ffad324e2aaafef691026fe15a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.015173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:18.028080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17b6e7ffad324e2aaafef691026fe15a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.028964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17b6e7ffad324e2aaafef691026fe15a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.029819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17b6e7ffad324e2aaafef691026fe15a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.059870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1236/2450] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:17 2022] DELETE /identity/v3/users/17b6e7ffad324e2aaafef691026fe15a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:18.064768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:18.065059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:18.065419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:18.084598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:18.086051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:18.158685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1215/2451] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:18.168469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:18.170139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:18.262775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6bf7e5-387f-48dc-863e-3927c9b52ab2 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:18.266545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f908-3770-434f-906c-6dcae8bc3b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:18.267204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f908-3770-434f-906c-6dcae8bc3b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:18.267512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f908-3770-434f-906c-6dcae8bc3b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:18.267826 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f908-3770-434f-906c-6dcae8bc3b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:18.287964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2f908-3770-434f-906c-6dcae8bc3b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:18.289600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2f908-3770-434f-906c-6dcae8bc3b9c service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:18.301700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1237/2452] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:18.682028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:18.683710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:18.711188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7b1a7f-b14f-4023-a3ad-8591c21604ec None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:18.717497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:18.718658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:18.719033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:18.719431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] PATH_INFO: `/v3/projects/daafa90f8b164d30ae44ba253470c82f` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:18.720850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=daafa90f8b164d30ae44ba253470c82f)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:18.724063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:18.782232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daafa90f8b164d30ae44ba253470c82f', 'request_id': 'req-22bb63db-a343-49ed-99c4-f21e5fabea04'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.793885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daafa90f8b164d30ae44ba253470c82f', 'request_id': 'req-22bb63db-a343-49ed-99c4-f21e5fabea04'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.802863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daafa90f8b164d30ae44ba253470c82f', 'request_id': 'req-22bb63db-a343-49ed-99c4-f21e5fabea04'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.816875 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] The token cache is being invalidate because project daafa90f8b164d30ae44ba253470c82f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:18.817666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.818376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-22bb63db-a343-49ed-99c4-f21e5fabea04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:18.819973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1216/2453] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:18 2022] DELETE /identity/v3/projects/daafa90f8b164d30ae44ba253470c82f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:18.824946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:18.825845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:18.826321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:18.863968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:18.865952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:19.058149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1238/2454] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:19.069002 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.069290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.069574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.102351 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:19.103676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:19.113730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1217/2455] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:19.118006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.130394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e457e-c101-4111-86a7-4ee720b413de None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.136552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.137670 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.137959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.138333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.139361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:19.139944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.141916 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.156848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1239/2456] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:44:19.164059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.177916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e05631-e143-42dd-915e-ed407764fc66 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.186476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.187421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.188039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.188292 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.189398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.191830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.210097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:19.230637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1218/2457] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:19 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:19.234857 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.245872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17340693-3a5b-4e6e-8bf8-4e578d391d8e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.332323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.333043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.333328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.333650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.334469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.336766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.353565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:19.370084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1240/2458] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:19 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:19.373914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.385577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a628a635-2952-4545-87ba-aff4a9ea0374 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.390649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.391391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.391678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.391960 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.392688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.394540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.405811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1219/2459] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:19.410423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.421936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdc4d2d-4735-42d8-bbb1-44e6d4efe700 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.427203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86bd61a-8adf-41b5-b771-bbd131ec43a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.427978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bd61a-8adf-41b5-b771-bbd131ec43a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.428273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bd61a-8adf-41b5-b771-bbd131ec43a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.428643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86bd61a-8adf-41b5-b771-bbd131ec43a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.429462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bd61a-8adf-41b5-b771-bbd131ec43a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.435708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86bd61a-8adf-41b5-b771-bbd131ec43a9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.450969 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1241/2460] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:19.455984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.470831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476b3b2-761b-40ba-91c7-3deb040c58e0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.478318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.480266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.480558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.480841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] PATH_INFO: `/v3/projects/88f35437b5f14f18bbf17eb24d9a8172/users/f85431938bf54587a8c0cd0798dde074/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.498927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88f35437b5f14f18bbf17eb24d9a8172, user_id=f85431938bf54587a8c0cd0798dde074, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.502759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.526045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1220/2461] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:19 2022] PUT /identity/v3/projects/88f35437b5f14f18bbf17eb24d9a8172/users/f85431938bf54587a8c0cd0798dde074/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:19.538807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b1002-7224-4797-a90b-3a97fb602876 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.539006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b1002-7224-4797-a90b-3a97fb602876 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.539408 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36b1002-7224-4797-a90b-3a97fb602876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.576962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c36b1002-7224-4797-a90b-3a97fb602876 None None] MFA Rules not processed for user `f85431938bf54587a8c0cd0798dde074`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:19.578535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c36b1002-7224-4797-a90b-3a97fb602876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:19.684261 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1242/2462] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:19.697551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:19.699597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:19.805198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea60d242-5f31-4519-9352-1171c1bf97c8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:19.809444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:19.810377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:19.810915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:19.811035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:19.820767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:19.822113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:19.835256 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1221/2463] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:23.958323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:23.958949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:23.958949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:23.987467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:23.989301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:24.044634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1243/2464] 10.222.0.43 () {58 vars in 967 bytes} [Fri Jun 24 08:44:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:24.058616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:24.060143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:24.133096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79689a9b-e15f-440b-b1cc-dc2630fa645c service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:24.138076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7158a3-689b-4ad2-babf-216184700490 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:24.138762 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7158a3-689b-4ad2-babf-216184700490 service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:24.139037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7158a3-689b-4ad2-babf-216184700490 service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:24.139352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7158a3-689b-4ad2-babf-216184700490 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:24.148637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7158a3-689b-4ad2-babf-216184700490 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:24.150570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7158a3-689b-4ad2-babf-216184700490 service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:24.161456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1222/2465] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:44:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:44:24.656640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:24.743705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7260bb-3b81-44b3-8966-ef865384f729 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:24.746325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:24.747391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:24.747734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:24.747959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] PATH_INFO: `/v3/users/f85431938bf54587a8c0cd0798dde074` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:24.749472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f85431938bf54587a8c0cd0798dde074)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:24.751653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:24.765971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:24.811497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f85431938bf54587a8c0cd0798dde074'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:24.821141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f85431938bf54587a8c0cd0798dde074'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:24.829476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f85431938bf54587a8c0cd0798dde074'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:24.830116 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f85431938bf54587a8c0cd0798dde074'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:24.830722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f85431938bf54587a8c0cd0798dde074'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:24.849356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1244/2466] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:24 2022] DELETE /identity/v3/users/f85431938bf54587a8c0cd0798dde074 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:24.860737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:24.860968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:24.861250 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:24.879224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:24.880486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:24.928016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:24.943348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1223/2467] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:24.952886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:24.954292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:25.020385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6f13df-6c42-4091-a16a-81b26245938d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.024139 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce023cb8-fb3f-454e-a89c-48e1638665fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.024794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce023cb8-fb3f-454e-a89c-48e1638665fb service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.025074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce023cb8-fb3f-454e-a89c-48e1638665fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.025346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce023cb8-fb3f-454e-a89c-48e1638665fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.034391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce023cb8-fb3f-454e-a89c-48e1638665fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.036372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce023cb8-fb3f-454e-a89c-48e1638665fb service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.045944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1245/2468] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:25.308180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.309657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:25.321751 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010aede-6750-426f-b02e-c7c79425fc52 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.326480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.327383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.327665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.327942 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] PATH_INFO: `/v3/projects/88f35437b5f14f18bbf17eb24d9a8172` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.329130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88f35437b5f14f18bbf17eb24d9a8172)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.331055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.377806 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88f35437b5f14f18bbf17eb24d9a8172', 'request_id': 'req-60740350-09d4-4b37-a3ad-032662a78351'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:25.385149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88f35437b5f14f18bbf17eb24d9a8172', 'request_id': 'req-60740350-09d4-4b37-a3ad-032662a78351'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:25.391881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88f35437b5f14f18bbf17eb24d9a8172', 'request_id': 'req-60740350-09d4-4b37-a3ad-032662a78351'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:25.402061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] The token cache is being invalidate because project 88f35437b5f14f18bbf17eb24d9a8172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:25.402348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:25.402987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-60740350-09d4-4b37-a3ad-032662a78351 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:25.404560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1224/2469] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:25 2022] DELETE /identity/v3/projects/88f35437b5f14f18bbf17eb24d9a8172 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:25.409944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.410234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.410512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.428057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:25.429256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:25.495379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1246/2470] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:25.503308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.503828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.503974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.531624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:25.532826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:25.541319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1225/2471] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:25.545877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.556241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44d3b6-17ec-4651-b79c-8631de3639eb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.561310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.562253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.562559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.562827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.563651 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:25.564038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.566235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.578946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1247/2472] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:44:25.584665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.595036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8ccd8-7985-4120-85ed-d450a2a9bf61 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.600710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.601479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.601951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.602291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.603329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.605296 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.635372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1226/2473] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:25 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:25.645021 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.655355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597b5066-9519-4da8-86fb-8953819eca60 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.673015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:25.716998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.717694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.717977 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.718254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.718944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.720910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.740934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1248/2474] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:25 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:44:25.745051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.756291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63a6f0-58ce-421b-9eeb-e1d8ec22957a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.762115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.762980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.763359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.763736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.764696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.766380 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.779265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1227/2475] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:25.782558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.793547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f844c4f5-f217-439f-ac0b-67423100e329 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.798155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f293d-f9e3-4ec9-91cb-05a36d96eb0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.798823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f293d-f9e3-4ec9-91cb-05a36d96eb0e admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.799111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f293d-f9e3-4ec9-91cb-05a36d96eb0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.799433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f293d-f9e3-4ec9-91cb-05a36d96eb0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.800715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f293d-f9e3-4ec9-91cb-05a36d96eb0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.802303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f293d-f9e3-4ec9-91cb-05a36d96eb0e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.814551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1249/2476] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:25.819000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.830036 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6111a3-1473-43a3-8a36-c27229a64ffd admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:25.834834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:25.835754 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.836037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.836312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] PATH_INFO: `/v3/projects/891980c3542d4223a15461bb7c5dcff9/users/7f65bea891fb461086d6a60177ff2d65/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.853530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891980c3542d4223a15461bb7c5dcff9, user_id=7f65bea891fb461086d6a60177ff2d65, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:25.856341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:25.873223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1228/2477] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:25 2022] PUT /identity/v3/projects/891980c3542d4223a15461bb7c5dcff9/users/7f65bea891fb461086d6a60177ff2d65/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:25.878606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:25.879112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:25.879474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:25.913349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:25.915060 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:25.984685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1250/2478] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:25.994610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:25.996340 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:26.058019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d5756-268e-4e5b-9f65-c2fafb6e5fb9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:26.062191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:26.062852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:26.063138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:26.063450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:26.073383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:26.075416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:26.085769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1229/2479] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:44:27.433152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:27.436491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:27.462766 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6799c-992e-47f1-a14c-15be01fd890b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:27.472649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:27.474376 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:27.474875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:27.475884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] PATH_INFO: `/v3/users/46097d36b6fc4df5b510d279a4648eb7` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:27.478018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46097d36b6fc4df5b510d279a4648eb7)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:27.482424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:27.578237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46097d36b6fc4df5b510d279a4648eb7'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:27.593168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46097d36b6fc4df5b510d279a4648eb7'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:27.606231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46097d36b6fc4df5b510d279a4648eb7'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:27.615047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46097d36b6fc4df5b510d279a4648eb7'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:27.615822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e0041742-168a-4718-94b6-68ee17c40f95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46097d36b6fc4df5b510d279a4648eb7'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:27.645258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1251/2480] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:27 2022] DELETE /identity/v3/users/46097d36b6fc4df5b510d279a4648eb7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:27.936848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:27.951464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73fdbbb-231a-4d23-b640-260afe2c37fe service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:27.970424 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:28.041110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.041993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.042265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.042545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] PATH_INFO: `/v3/projects/5e6d51dece9847669e9a666633ce634d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.044061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e6d51dece9847669e9a666633ce634d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.045974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.095729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e6d51dece9847669e9a666633ce634d', 'request_id': 'req-71baf1f6-3643-4378-bc92-23762e7b464e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.104550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e6d51dece9847669e9a666633ce634d', 'request_id': 'req-71baf1f6-3643-4378-bc92-23762e7b464e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.111107 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e6d51dece9847669e9a666633ce634d', 'request_id': 'req-71baf1f6-3643-4378-bc92-23762e7b464e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.122197 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] The token cache is being invalidate because project 5e6d51dece9847669e9a666633ce634d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:28.122485 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.123164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-71baf1f6-3643-4378-bc92-23762e7b464e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.124759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1230/2481] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:27 2022] DELETE /identity/v3/projects/5e6d51dece9847669e9a666633ce634d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:28.131446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.131936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.132233 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.152158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:28.153416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:28.159402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:28.246005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1252/2482] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:28.253947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.254325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.254701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.274440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:28.275890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:28.285190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1231/2483] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:28.291892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.302389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e1b2e-6192-408c-beb9-efb0b4c653d0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.306964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.307650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.307934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.308206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.308994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:28.309297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.310887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.322355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1253/2484] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:44:28.326759 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.343529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2977c7-aafb-47c1-9d80-58eb62b78606 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.348878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.349650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.349939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.350215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.350913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.353248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.383231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1232/2485] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:28 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:28.387471 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.398523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe4cc9-202a-4c1f-95cf-910291b43ebf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.464325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.465040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.465329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.465784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.466529 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.468554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.496861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1254/2486] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:28 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:28.502780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.522420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b06b1c-bc1a-4524-bda7-ed98ec7edc1f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.531128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.532350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.532974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.533432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.535085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.536939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.549352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1233/2487] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:28.554771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.567763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28d160-f3f9-4bd0-b3ae-eb6e48273cb2 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.575322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.576897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.577269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.577562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.578363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.580628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.592483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1255/2488] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:28.596615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.607488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39cce1-2914-457d-ad5a-caf7d1ba844b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.614252 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac896d0b-627e-4eb8-a65c-a2d4022ada28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.615721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac896d0b-627e-4eb8-a65c-a2d4022ada28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.616178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac896d0b-627e-4eb8-a65c-a2d4022ada28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.616438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac896d0b-627e-4eb8-a65c-a2d4022ada28 admin admin] PATH_INFO: `/v3/projects/f1498e5a0d094306a02a9c2e039bb534/users/6e61a35683cc4c589ed9809c4fe001b1/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.637408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac896d0b-627e-4eb8-a65c-a2d4022ada28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1498e5a0d094306a02a9c2e039bb534, user_id=6e61a35683cc4c589ed9809c4fe001b1, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.642597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac896d0b-627e-4eb8-a65c-a2d4022ada28 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.659728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1234/2489] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:28 2022] PUT /identity/v3/projects/f1498e5a0d094306a02a9c2e039bb534/users/6e61a35683cc4c589ed9809c4fe001b1/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:44:28.673384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.675924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.676216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.676501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.679007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:28.705108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bfa06-569f-4672-93ca-885a2f6ee795 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.717495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] MFA Rules not processed for user `6e61a35683cc4c589ed9809c4fe001b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:28.719690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:28.756833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:28.806326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1235/2490] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:28.812550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.813254 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.813550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.813829 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] PATH_INFO: `/v3/users/3b5932082fd34372ad6c205e43538166` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.815146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5932082fd34372ad6c205e43538166)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.818149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:28.819115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.819625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:28.847876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:28.895683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5932082fd34372ad6c205e43538166'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.905177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5932082fd34372ad6c205e43538166'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.918535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5932082fd34372ad6c205e43538166'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.920223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5932082fd34372ad6c205e43538166'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.921041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6ddf453d-7ddb-49c6-ab70-ecc12498c4b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5932082fd34372ad6c205e43538166'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:28.935133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f6de9-0fbe-4561-bdd3-52b57e92d537 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:28.942567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:28.943649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.944058 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.944426 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.947982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:28.952190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1256/2491] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:28 2022] DELETE /identity/v3/users/3b5932082fd34372ad6c205e43538166 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:28.958851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:28.963137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:28.963522 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:28.969691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:28.971310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:28.983098 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1236/2492] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:44:28.990580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:28.992247 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:29.013884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:29.015486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:29.031057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7110a7-0777-4cd8-a93f-1c7e2f364486 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:29.115676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:29.118558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:29.118868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:29.119146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] PATH_INFO: `/v3/users/6e61a35683cc4c589ed9809c4fe001b1` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:29.122199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e61a35683cc4c589ed9809c4fe001b1)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:29.126438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:29.169362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1257/2493] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:29.185276 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:29.187020 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:29.228888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e61a35683cc4c589ed9809c4fe001b1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:29.240589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e61a35683cc4c589ed9809c4fe001b1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:29.259430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e61a35683cc4c589ed9809c4fe001b1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:29.262230 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:29.272344 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e61a35683cc4c589ed9809c4fe001b1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:29.273938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-79a95c95-6630-47d1-bb3f-a71ec4136c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e61a35683cc4c589ed9809c4fe001b1'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:29.282628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1237/2494] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:44:29 2022] DELETE /identity/v3/users/6e61a35683cc4c589ed9809c4fe001b1 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:29.290964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:29.291783 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:29.292073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:29.318560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063dccd0-4220-4bad-888c-46d75cb3c1a3 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:29.349891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:29.354932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:29.425360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:29.426561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:29.427267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:29.427552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:29.430173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:29.437837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:29.471139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1238/2495] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:29.485597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:29.488368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:29.511504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09201d78-9976-44b4-8242-2c1efa4a69d8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:29.516880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:29.518510 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:29.519830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:29.520293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:29.521825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:29.523704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:29.533735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:29.535612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:29.537445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1258/2496] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:44:29.550504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1239/2497] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:30.004181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:30.005904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:30.027987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2228f3-7521-435f-8c48-ffd7e9845b86 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:30.041341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:30.045227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.046058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.047966 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] PATH_INFO: `/v3/projects/6a5c5193e1994a57a250068bb347b259` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.050308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a5c5193e1994a57a250068bb347b259)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:30.053906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:30.181479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:30.182920 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:30.230888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62fd3c2-304a-4182-9ec7-0258e99b4ac5 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:30.242436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a5c5193e1994a57a250068bb347b259', 'request_id': 'req-209eb4db-dcb2-4048-a2b6-430066fb3d9c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.252511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a5c5193e1994a57a250068bb347b259', 'request_id': 'req-209eb4db-dcb2-4048-a2b6-430066fb3d9c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.273505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a5c5193e1994a57a250068bb347b259', 'request_id': 'req-209eb4db-dcb2-4048-a2b6-430066fb3d9c'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.296948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] The token cache is being invalidate because project 6a5c5193e1994a57a250068bb347b259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:30.297180 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.298947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-209eb4db-dcb2-4048-a2b6-430066fb3d9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.306057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1259/2498] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:30 2022] DELETE /identity/v3/projects/6a5c5193e1994a57a250068bb347b259 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:30.312291 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04a758-7831-4b49-a1eb-8367c3264f9e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.312641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04a758-7831-4b49-a1eb-8367c3264f9e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.312935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04a758-7831-4b49-a1eb-8367c3264f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.375887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1c04a758-7831-4b49-a1eb-8367c3264f9e None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:30.380960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1c04a758-7831-4b49-a1eb-8367c3264f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:30.419293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:30.420301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.423150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.423591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] PATH_INFO: `/v3/projects/f1498e5a0d094306a02a9c2e039bb534` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.426374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1498e5a0d094306a02a9c2e039bb534)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:30.433634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:30.563165 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1498e5a0d094306a02a9c2e039bb534', 'request_id': 'req-959efa5b-e669-4285-a240-4b3934e05036'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.582062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1260/2499] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:30.583232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1498e5a0d094306a02a9c2e039bb534', 'request_id': 'req-959efa5b-e669-4285-a240-4b3934e05036'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.583888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.584190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.584486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.596261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1498e5a0d094306a02a9c2e039bb534', 'request_id': 'req-959efa5b-e669-4285-a240-4b3934e05036'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.610157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:30.612157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:30.615221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] The token cache is being invalidate because project f1498e5a0d094306a02a9c2e039bb534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:44:30.615659 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.618660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-959efa5b-e669-4285-a240-4b3934e05036 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:44:30.624736 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1240/2500] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:44:30 2022] DELETE /identity/v3/projects/f1498e5a0d094306a02a9c2e039bb534 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:44:30.626267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304118a0-544d-423b-b596-6316526beaea None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.630368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304118a0-544d-423b-b596-6316526beaea None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.630581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304118a0-544d-423b-b596-6316526beaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.638019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:30.683206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-304118a0-544d-423b-b596-6316526beaea None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:30.687707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-304118a0-544d-423b-b596-6316526beaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:30.697815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:30.723052 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1261/2501] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:30.726216 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:30.756188 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1f008-27f4-4818-998d-dfaf498daf96 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:30.760949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:30.761909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.761909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.762149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.762902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:30.763190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:30.765176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:30.794354 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1262/2502] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:44:30.795267 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1241/2503] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:30.798696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:30.804727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.805123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.805525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.815735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6d0a3e-8e89-41c1-8422-e001c135f8a2 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:30.822704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:30.823637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.824100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.824535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.825547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:30.828270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:30.844582 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:30.846076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:30.863112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1242/2504] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:30.870212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:30.874993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1263/2505] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:30 2022] POST /identity/v3/projects => generated 363 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:30.879171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:30.893867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7490f-fc38-4e1f-a7ec-81815cfb6cfd None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:30.993986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:30.995410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:30.995832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:30.996333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:30.997242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:44:30.997653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.000496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.020455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc61c1b-fcf2-4a56-9fad-848d22f9bf1f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.032243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1243/2506] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:44:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 08:44:31.036996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.041968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.042934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.042934 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.043504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.044214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.050727 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.057786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27a498-5e57-4832-b39e-0e05b3bd0eb4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.071459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.072459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.074539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.075045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.077350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.085405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.093338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1264/2507] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:30 2022] POST /identity/v3/users => generated 282 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:31.097506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.117605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c17f57e-bec5-40f5-a1f6-bcaf6ed45be1 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.129962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.130973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.133814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.134389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.138190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.144814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.146624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1244/2508] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/projects => generated 391 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:31.148749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.180213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1265/2509] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:44:31.180213 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.209775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd55361a-09f6-4c73-a68a-e6ce96fc1b68 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.311371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9f775-f611-401c-9604-e64a9bfca30b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.324176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.324981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.326068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.326622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.328636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.333264 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.351321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.352160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.352551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.352952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.354506 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.357812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.372967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1245/2510] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/users => generated 296 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:31.377293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.382010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1266/2511] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:31.385248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.394441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:31.402585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c62bb-fbf4-442b-9e81-5f48cb4f22e4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.403032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca06400d-8996-419b-a5ba-8b593b3966bd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.410314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.411739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.412140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.412516 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] PATH_INFO: `/v3/projects/9b9358f9dffd4721b65d357335213f71/users/9b0239306056478493f31c8ea4aa8fdf/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.416634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe88ef0-2f82-43c1-8aed-d0cd58962b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.417318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe88ef0-2f82-43c1-8aed-d0cd58962b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.417521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:31.417652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe88ef0-2f82-43c1-8aed-d0cd58962b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.417867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe88ef0-2f82-43c1-8aed-d0cd58962b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.419015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe88ef0-2f82-43c1-8aed-d0cd58962b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.423611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe88ef0-2f82-43c1-8aed-d0cd58962b6c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.442322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9358f9dffd4721b65d357335213f71, user_id=9b0239306056478493f31c8ea4aa8fdf, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.446499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.452432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1246/2512] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:44:31.459245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.459630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.459903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.463852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1267/2513] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:31 2022] PUT /identity/v3/projects/9b9358f9dffd4721b65d357335213f71/users/9b0239306056478493f31c8ea4aa8fdf/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:31.465006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.530312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] MFA Rules not processed for user `7f65bea891fb461086d6a60177ff2d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:31.535538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:31.559793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251686b6-f47b-4c43-96d9-30b29aaa7203 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.566532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86feadd1-92ee-4601-9513-a7d05adbea73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.568023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feadd1-92ee-4601-9513-a7d05adbea73 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.568691 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feadd1-92ee-4601-9513-a7d05adbea73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.569582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86feadd1-92ee-4601-9513-a7d05adbea73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.570535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86feadd1-92ee-4601-9513-a7d05adbea73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.572468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86feadd1-92ee-4601-9513-a7d05adbea73 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.607729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1268/2514] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:31 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:31.609335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.609622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.609901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.682725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] MFA Rules not processed for user `9b0239306056478493f31c8ea4aa8fdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:31.689349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:31.740368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1247/2515] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:31.741371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.757413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb773f-2b82-443b-aca9-d15ed1f1e13b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.767544 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1582c-28c0-415d-b895-5a67f4f248c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.768455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1582c-28c0-415d-b895-5a67f4f248c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.769765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1582c-28c0-415d-b895-5a67f4f248c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.770051 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1582c-28c0-415d-b895-5a67f4f248c5 admin admin] PATH_INFO: `/v3/projects/e084adfeeac94248860c453d8e56b629/users/9fa86472f2db4436ad7f9f8a240c804c/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.790429 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1582c-28c0-415d-b895-5a67f4f248c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e084adfeeac94248860c453d8e56b629, user_id=9fa86472f2db4436ad7f9f8a240c804c, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.793596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1582c-28c0-415d-b895-5a67f4f248c5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.814636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1269/2516] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:31.815178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1248/2517] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:31 2022] PUT /identity/v3/projects/e084adfeeac94248860c453d8e56b629/users/9fa86472f2db4436ad7f9f8a240c804c/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:31.821656 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.825763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:31.828371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.828819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.829356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.844448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1901e50-8390-4611-bf52-c60e8e142b22 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:31.870173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] MFA Rules not processed for user `9fa86472f2db4436ad7f9f8a240c804c`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:31.871509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:31.930904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:31.931637 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:31.931919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:31.932197 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:31.932897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:31.935122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:31.949994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:31.968801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1270/2518] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/projects => generated 379 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:31.968801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.969277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:31.982287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1249/2519] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:31.983572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:31.996714 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68aa74-2b1d-4c33-8744-e53cc2a851df None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.058779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:32.065360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be94dd55-bb19-4a87-93ee-e1b6e2012c34 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.073671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.075119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.075550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.075902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.080359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.082092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.082542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.083006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.083951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.086189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.115428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1250/2520] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:44:31 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:44:32.116388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:32.118195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:32.132697 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656fe5e2-1118-4f5b-a18e-b80ba98d609c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.140185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.141387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.142100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.142393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.148657 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.150128 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.161175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1271/2521] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:32.172365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:32.217799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8926de91-5d41-4377-bbdc-bd8394e25164 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.237226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.237974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.238257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.238536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.239524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.241145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.254744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.256631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.273459 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1251/2522] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:32.278478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1272/2523] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:32.283282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:32.297212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc11d88-5eee-4047-aec4-6cbe34f9659b service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.307317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc35f1-112a-4327-acdc-20bd1a798ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.309277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc35f1-112a-4327-acdc-20bd1a798ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.309478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc35f1-112a-4327-acdc-20bd1a798ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.309907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc35f1-112a-4327-acdc-20bd1a798ae4 admin admin] PATH_INFO: `/v3/projects/a5bc8829106841d6accb393f68d1f594/users/8ee691ec36424bacadf4b0d26dc1edf7/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.334037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc35f1-112a-4327-acdc-20bd1a798ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bc8829106841d6accb393f68d1f594, user_id=8ee691ec36424bacadf4b0d26dc1edf7, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.336860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bc35f1-112a-4327-acdc-20bd1a798ae4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.361772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1252/2524] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:32 2022] PUT /identity/v3/projects/a5bc8829106841d6accb393f68d1f594/users/8ee691ec36424bacadf4b0d26dc1edf7/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:32.363937 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:32.377565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98153fe9-3bc0-4f14-8838-ddc62693eea4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.419954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.420349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.420810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.450996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] MFA Rules not processed for user `9abdce8004e24e81bd88bee757d8ec93`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:32.452658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:32.477258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.478068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.478437 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.479418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.482285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.487929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.511721 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1273/2525] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:44:32 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:44:32.518120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:32.542211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb33e0-9262-467f-aeaf-be2e01583c0f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:32.549487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1253/2526] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:44:32.555877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971def8-e2bc-4d03-8e16-a705d00f0745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:32.556824 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971def8-e2bc-4d03-8e16-a705d00f0745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:32.557137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971def8-e2bc-4d03-8e16-a705d00f0745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:32.557507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971def8-e2bc-4d03-8e16-a705d00f0745 admin admin] PATH_INFO: `/v3/projects/a5bc8829106841d6accb393f68d1f594/users/8ee691ec36424bacadf4b0d26dc1edf7/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:32.559862 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971def8-e2bc-4d03-8e16-a705d00f0745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5bc8829106841d6accb393f68d1f594, user_id=8ee691ec36424bacadf4b0d26dc1edf7, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:32.562803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971def8-e2bc-4d03-8e16-a705d00f0745 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:32.576456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1274/2527] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:44:32 2022] PUT /identity/v3/projects/a5bc8829106841d6accb393f68d1f594/users/8ee691ec36424bacadf4b0d26dc1edf7/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:44:35.692554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:35.694803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:35.839701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d98ca91-b5ea-4a6e-b0be-3f5f5461543b None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:35.859721 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:36.016528 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:36.017206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:36.017493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:36.017771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:36.037833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:36.199116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:36.200828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:36.236144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1254/2528] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:39.491370 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:39.491859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:39.491959 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:39.523356 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] MFA Rules not processed for user `8ee691ec36424bacadf4b0d26dc1edf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:44:39.524653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:39.562340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:39.728125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1275/2529] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:44:39 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:44:39.767367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:39.768919 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:39.944213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c8fb92-901b-4ec7-b255-cb0e9c5f0703 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:39.951314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:39.952112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:39.952494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:39.952863 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:39.997563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:40.031167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:40.032952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:40.046529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1255/2530] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:40.246591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:44:40.248244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:40.425296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b0e348-f132-49c3-8471-4880d1b8459b None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:44:40.439532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:44:40.441210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:44:40.441210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:44:40.441446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:44:40.448151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:44:40.475350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:44:40.476967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab47e1c-02f3-4cd4-8664-c0b9c06fdd67 service placement] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:44:40.506100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1276/2531] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:44:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:44:50.040673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=12 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:44:50.496107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=66 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:06.157597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:06.223775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb2109-c087-47ff-9a32-ba89330889ed service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:06.228536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:06.229419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:06.229699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:06.229979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] PATH_INFO: `/v3/users/9fa86472f2db4436ad7f9f8a240c804c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:06.231226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa86472f2db4436ad7f9f8a240c804c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:06.233355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:06.296833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa86472f2db4436ad7f9f8a240c804c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.304386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa86472f2db4436ad7f9f8a240c804c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.317606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa86472f2db4436ad7f9f8a240c804c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.325509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa86472f2db4436ad7f9f8a240c804c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.326408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa86472f2db4436ad7f9f8a240c804c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.332590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1256/2532] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:45:06 2022] DELETE /identity/v3/users/9fa86472f2db4436ad7f9f8a240c804c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:45:06.337932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:06.338476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:06.338476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:06.356671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:06.357993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:06.426647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1277/2533] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:06.436208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:06.437837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:06.502621 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74aba1-3346-480e-b923-f56c83792442 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:06.506210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:06.506855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:06.507135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:06.507436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:06.517954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:06.519414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:06.528935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1257/2534] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:45:06.817508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:06.818944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:06.832267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc2742e-23dd-4565-935f-8d4e82f17cb0 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:06.837022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:06.837897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:06.838179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:06.838456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] PATH_INFO: `/v3/projects/e084adfeeac94248860c453d8e56b629` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:06.839667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e084adfeeac94248860c453d8e56b629)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:06.841676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:06.888742 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e084adfeeac94248860c453d8e56b629', 'request_id': 'req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.894265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e084adfeeac94248860c453d8e56b629', 'request_id': 'req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.900395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e084adfeeac94248860c453d8e56b629', 'request_id': 'req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.912133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] The token cache is being invalidate because project e084adfeeac94248860c453d8e56b629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:45:06.912422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.913038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9bd0afea-ae6b-4d85-bf2e-e457566f9d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:06.914583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1278/2535] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:45:06 2022] DELETE /identity/v3/projects/e084adfeeac94248860c453d8e56b629 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:45:16.524630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:16.909106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:18.654073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:18.655475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:18.667080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b439931-4175-4d7f-a60d-61d48e298fb6 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:18.726679 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:18.727371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:18.727656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:18.727931 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:18.730053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:18.794142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:18.795620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:18.806647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1258/2536] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:18 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:45:20.879025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835632f-b6e7-44d2-8112-8aa4cd2ed0cb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:20.879596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835632f-b6e7-44d2-8112-8aa4cd2ed0cb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:20.879774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835632f-b6e7-44d2-8112-8aa4cd2ed0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:20.898901 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-0835632f-b6e7-44d2-8112-8aa4cd2ed0cb None None] MFA Rules not processed for user `7f65bea891fb461086d6a60177ff2d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:20.900173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0835632f-b6e7-44d2-8112-8aa4cd2ed0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:20.961474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1279/2537] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:45:20 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:28.801706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=50 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:30.953472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=25 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:39.535411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:39.536810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:39.607070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53f6ad-722c-4545-99a8-2a13c181d6c0 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:39.612245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:39.613111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:39.613394 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:39.613670 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] PATH_INFO: `/v3/users/9b0239306056478493f31c8ea4aa8fdf` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:39.614866 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0239306056478493f31c8ea4aa8fdf)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:39.616824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:39.663232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0239306056478493f31c8ea4aa8fdf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:39.670657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0239306056478493f31c8ea4aa8fdf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:39.684486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0239306056478493f31c8ea4aa8fdf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:39.691474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0239306056478493f31c8ea4aa8fdf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:39.692180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0239306056478493f31c8ea4aa8fdf'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:39.698253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1259/2538] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:45:39 2022] DELETE /identity/v3/users/9b0239306056478493f31c8ea4aa8fdf => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:45:39.704982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:39.704982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:39.704982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:39.722726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:39.724373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:39.782266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1280/2539] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:39.792357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:39.793746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:39.851048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e43f666-2fd6-45e5-a3f3-c0b991a60a86 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:39.854686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:39.855361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:39.855647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:39.855924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:39.865326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:39.866757 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:39.876075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1260/2540] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:45:40.132022 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:40.133439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:40.146053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a1e4e0-c55d-4e1d-930d-55fb01922d1a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:40.151613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:40.152662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:40.152951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:40.153349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] PATH_INFO: `/v3/projects/9b9358f9dffd4721b65d357335213f71` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:40.154633 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9358f9dffd4721b65d357335213f71)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:40.157284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:40.206095 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9358f9dffd4721b65d357335213f71', 'request_id': 'req-310104df-1fd4-4625-9907-074fb9bc4d9a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:40.211668 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9358f9dffd4721b65d357335213f71', 'request_id': 'req-310104df-1fd4-4625-9907-074fb9bc4d9a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:40.217843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9358f9dffd4721b65d357335213f71', 'request_id': 'req-310104df-1fd4-4625-9907-074fb9bc4d9a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:40.236001 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] The token cache is being invalidate because project 9b9358f9dffd4721b65d357335213f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:45:40.236301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:40.237000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-310104df-1fd4-4625-9907-074fb9bc4d9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:45:40.239092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1281/2541] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:45:40 2022] DELETE /identity/v3/projects/9b9358f9dffd4721b65d357335213f71 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:45:40.566028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:40.574442 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7f3a3-ff4b-489b-8fe0-d8ebdc03fa80 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:40.592664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:40.623782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:40.624414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:40.624698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:40.624973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:40.626725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:40.686508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:40.687976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:40.697343 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1261/2542] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:45:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:45:50.232692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:50.693130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=40 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:45:53.573595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efbf31-41ce-42f0-a651-e316b8b66fd0 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:53.574141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efbf31-41ce-42f0-a651-e316b8b66fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:53.574556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97efbf31-41ce-42f0-a651-e316b8b66fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:53.627697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-97efbf31-41ce-42f0-a651-e316b8b66fd0 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:45:53.628836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-97efbf31-41ce-42f0-a651-e316b8b66fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:53.691632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1282/2543] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:45:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:45:53.737108 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:45:53.738516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:45:53.831550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a18af5e-9c5f-4f84-82d4-e09ba7eea9cb service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:45:53.836644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c435eb3-0b9c-4290-8cfb-f20047080b2b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:45:53.837414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c435eb3-0b9c-4290-8cfb-f20047080b2b service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:45:53.837699 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c435eb3-0b9c-4290-8cfb-f20047080b2b service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:45:53.837973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c435eb3-0b9c-4290-8cfb-f20047080b2b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:45:53.846996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c435eb3-0b9c-4290-8cfb-f20047080b2b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:45:53.848644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c435eb3-0b9c-4290-8cfb-f20047080b2b service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:45:53.857135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1262/2544] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:45:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:46:03.690556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=40 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:03.853768 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=41 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:09.832303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4830450-d370-4911-9921-83a69f322e41 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:09.832866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4830450-d370-4911-9921-83a69f322e41 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:09.833055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4830450-d370-4911-9921-83a69f322e41 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:09.834537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1283/2545] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:46:09.841182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bf1ea-9b3e-44ef-8283-68851a1e6a6c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:09.841641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bf1ea-9b3e-44ef-8283-68851a1e6a6c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:09.841779 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bf1ea-9b3e-44ef-8283-68851a1e6a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:09.879382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-225bf1ea-9b3e-44ef-8283-68851a1e6a6c None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:46:09.880599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-225bf1ea-9b3e-44ef-8283-68851a1e6a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:09.947540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1263/2546] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:46:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:46:09.964338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4830450-d370-4911-9921-83a69f322e41 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:09.966181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e4830450-d370-4911-9921-83a69f322e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:10.045336 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4830450-d370-4911-9921-83a69f322e41 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:10.049520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1829da-7002-4794-a219-0546ab13f319 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:10.050160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1829da-7002-4794-a219-0546ab13f319 service glance] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.050440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1829da-7002-4794-a219-0546ab13f319 service glance] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.050712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1829da-7002-4794-a219-0546ab13f319 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.060760 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1829da-7002-4794-a219-0546ab13f319 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:10.062134 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1829da-7002-4794-a219-0546ab13f319 service glance] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:10.073352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1284/2547] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:10.138752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84c70d-86f9-4f3f-b3f9-aba4c46df09e None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.139038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84c70d-86f9-4f3f-b3f9-aba4c46df09e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.139341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84c70d-86f9-4f3f-b3f9-aba4c46df09e None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.140725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1264/2548] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:46:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:46:10.145613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65ac00-abca-4bc3-9b0e-3d8d9abddf95 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.145952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65ac00-abca-4bc3-9b0e-3d8d9abddf95 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.146234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65ac00-abca-4bc3-9b0e-3d8d9abddf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.164287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-3c65ac00-abca-4bc3-9b0e-3d8d9abddf95 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:46:10.165491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3c65ac00-abca-4bc3-9b0e-3d8d9abddf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:10.172258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1285/2549] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:46:10.184242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84c70d-86f9-4f3f-b3f9-aba4c46df09e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:10.185951 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8c84c70d-86f9-4f3f-b3f9-aba4c46df09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:10.199688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84c70d-86f9-4f3f-b3f9-aba4c46df09e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:10.203341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4eedf-1f7e-408b-a70b-b40cd0cf0286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:10.203976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4eedf-1f7e-408b-a70b-b40cd0cf0286 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.204257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4eedf-1f7e-408b-a70b-b40cd0cf0286 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.204561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4eedf-1f7e-408b-a70b-b40cd0cf0286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.211946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4eedf-1f7e-408b-a70b-b40cd0cf0286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:10.213404 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c4eedf-1f7e-408b-a70b-b40cd0cf0286 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:10.221641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1265/2550] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:10.303122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787741fb-b869-4f8b-b7dc-3c3a6f5cf194 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.303560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787741fb-b869-4f8b-b7dc-3c3a6f5cf194 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.303929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787741fb-b869-4f8b-b7dc-3c3a6f5cf194 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.305456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1286/2551] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:46:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:46:10.310750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e709f-4c8d-4352-8774-6e8ffc0c0274 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.311037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e709f-4c8d-4352-8774-6e8ffc0c0274 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.311342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e709f-4c8d-4352-8774-6e8ffc0c0274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.329366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7c0e709f-4c8d-4352-8774-6e8ffc0c0274 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:46:10.330551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7c0e709f-4c8d-4352-8774-6e8ffc0c0274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:10.339593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1266/2552] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:46:10 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:46:10.350218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787741fb-b869-4f8b-b7dc-3c3a6f5cf194 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:10.352008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-787741fb-b869-4f8b-b7dc-3c3a6f5cf194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:10.365786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787741fb-b869-4f8b-b7dc-3c3a6f5cf194 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:10.369330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:10.369961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:10.370246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:10.370524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:10.381653 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:10.383067 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:10.393941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1287/2553] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:13.110062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:13.110522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:13.110626 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:13.128035 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:46:13.129238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:13.136492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1267/2554] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:46:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:46:13.148661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:13.160382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b7085-632a-44da-8be5-92ef988ed6a1 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:13.165334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:13.165975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:13.166257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:13.166561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:13.175102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:13.176722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:13.185074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1288/2555] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:46:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:46:23.122823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=49 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:23.181680 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=50 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:33.346224 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:33.355453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1babae77-afa7-4ed8-8336-7e5d235f2a97 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:33.358930 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:33.359597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:33.359876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:33.360149 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:33.362389 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:33.375451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:33.377469 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f78c126-4fae-4748-b379-9db645a8d49e service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:33.388218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1268/2556] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:33.831607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:33.841371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f91d5f-743f-41a5-8a1c-00c8b8882ef1 service swift] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:33.845194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:33.845871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:33.846178 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:33.846483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:33.848319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:33.862112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:33.863552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:33.874416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1289/2557] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:46:41.967837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666521db-485e-4508-83ff-de0ee2598730 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:41.968308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666521db-485e-4508-83ff-de0ee2598730 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:41.968402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666521db-485e-4508-83ff-de0ee2598730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:41.986776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-666521db-485e-4508-83ff-de0ee2598730 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:46:41.988068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-666521db-485e-4508-83ff-de0ee2598730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:41.995117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1269/2558] 10.222.0.43 () {58 vars in 967 bytes} [Fri Jun 24 08:46:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:46:42.007189 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:42.018500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef915-6abe-4934-a227-37a84bbd4f89 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:42.023485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:42.024129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:42.024415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:42.024813 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:42.033840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:42.035474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:42.043549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1290/2559] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:46:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:46:51.981087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=9 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:52.040323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=12 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:46:54.056938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666521db-485e-4508-83ff-de0ee2598730 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:46:54.058596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-666521db-485e-4508-83ff-de0ee2598730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:54.074240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666521db-485e-4508-83ff-de0ee2598730 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:46:54.080169 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:46:54.080805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:46:54.081087 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:46:54.081365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:46:54.083075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:46:54.139923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:46:54.141575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5b922-2440-4d89-be35-6c37aee0f799 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:46:54.151229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1270/2560] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:46:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:04.146966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=28 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:20.004752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:20.014084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ab90b0-1279-446d-a1ee-19fbcb7b59bf service swift] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:20.017609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:20.018239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:20.018536 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:20.018793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:20.020551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:20.029836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:20.031270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:20.039758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1291/2561] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:23.141383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:23.141926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:23.141926 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:23.159032 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] MFA Rules not processed for user `9abdce8004e24e81bd88bee757d8ec93`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:23.160253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:23.167430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1271/2562] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:47:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:29.144302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:29.145987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:29.163323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4744e3-989f-4778-8066-9332973da539 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:29.171349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:29.171349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:29.171349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:29.171349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:29.173882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:29.186373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:29.188016 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:29.199298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1292/2563] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:33.153936 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:39.194803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=12 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:47:49.795452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:49.808818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5925d79f-e0b9-44bb-aced-5802fc8a118a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:49.814184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:49.814826 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:49.815112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:49.815419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:49.817182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:49.875157 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:49.876793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dca2cb2-3243-4c30-81ee-d2ed8082f7e7 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:49.886355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1272/2564] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:47:56.635965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:56.649421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ef344-9247-4312-9dd9-fbd9f3045974 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:56.655226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:56.655962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:56.656245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:56.656520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:56.659023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:56.672361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:56.674623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:56.686366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1293/2565] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:47:59.490065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535850-385d-4b57-b54f-298d87837141 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:59.490601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535850-385d-4b57-b54f-298d87837141 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:59.490601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec535850-385d-4b57-b54f-298d87837141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:59.509083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ec535850-385d-4b57-b54f-298d87837141 None None] MFA Rules not processed for user `12f0ce0a293f4114a90098985e5f21ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:47:59.512017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ec535850-385d-4b57-b54f-298d87837141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:59.519227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1273/2566] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:47:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:47:59.823274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:47:59.843722 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca69850-faee-46e4-bb9e-12908ebfea9b service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:47:59.848042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:47:59.848907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:47:59.849212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:47:59.849482 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:47:59.851287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:47:59.862111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:47:59.863559 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36f00df-2233-4df7-bfdc-8e4d2f4b4ac6 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:47:59.871635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1294/2567] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:47:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:01.225873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535850-385d-4b57-b54f-298d87837141 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:01.227300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ec535850-385d-4b57-b54f-298d87837141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:01.302673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec535850-385d-4b57-b54f-298d87837141 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:01.308399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:01.309790 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:01.310074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:01.310347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:01.312285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:01.325039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:01.327039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:01.339302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1274/2568] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:01.341897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:01.342385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:01.342500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:01.363070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:01.364323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:01.371825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1295/2569] 10.222.0.43 () {58 vars in 967 bytes} [Fri Jun 24 08:48:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:01.421282 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:01.434479 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dfba6e-c9d2-424e-b4ba-5727426def90 service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:01.439522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:01.440155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:01.440438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:01.440738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:01.448750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:01.450317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:01.458225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1275/2570] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:48:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:48:01.800443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:01.810023 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb37f0d9-1a5e-40df-adf9-ebe729ad5508 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:01.813583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:01.814236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:01.814518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:01.814794 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:01.816584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:01.879328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:01.880764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:01.890470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1296/2571] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:04.963195 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:04.987190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f55eb56-3ff1-4e3a-bb35-febcd37801f2 service swift] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:04.993377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:04.994355 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:04.994886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:04.995561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:04.998192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:05.021242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:05.023821 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:05.041889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1276/2572] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:08.944890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:08.946367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:08.963600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767cee10-973d-460f-961b-96e260b687f4 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:08.970201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:08.971586 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:08.972032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:08.972449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] PATH_INFO: `/v3/users/4dbbf627f3bf4ffb87c6e23fac0b5002` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:08.974446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dbbf627f3bf4ffb87c6e23fac0b5002)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:08.977825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:09.032995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dbbf627f3bf4ffb87c6e23fac0b5002'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.039695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dbbf627f3bf4ffb87c6e23fac0b5002'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.046564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dbbf627f3bf4ffb87c6e23fac0b5002'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.047453 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dbbf627f3bf4ffb87c6e23fac0b5002'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.048061 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dbbf627f3bf4ffb87c6e23fac0b5002'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.065708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1297/2573] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:08 2022] DELETE /identity/v3/users/4dbbf627f3bf4ffb87c6e23fac0b5002 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:09.249141 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:09.261140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620edcb-e22d-4ef1-9b7a-7c412019ee40 service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:09.316214 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:09.323688 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:09.324835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:09.325146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:09.325452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] PATH_INFO: `/v3/projects/a862170421a346088ab15bc89c9a5716` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:09.326735 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a862170421a346088ab15bc89c9a5716)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:09.328844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:09.374295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a862170421a346088ab15bc89c9a5716', 'request_id': 'req-49c838f3-1be2-48b5-9f9c-7a317c404232'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.382005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a862170421a346088ab15bc89c9a5716', 'request_id': 'req-49c838f3-1be2-48b5-9f9c-7a317c404232'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.388907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a862170421a346088ab15bc89c9a5716', 'request_id': 'req-49c838f3-1be2-48b5-9f9c-7a317c404232'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.399976 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] The token cache is being invalidate because project a862170421a346088ab15bc89c9a5716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:09.400261 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.401006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.402660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1277/2574] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:09 2022] DELETE /identity/v3/projects/a862170421a346088ab15bc89c9a5716 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:09.406598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:09.409602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:09.423242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617b11e4-d503-4f6b-9914-bec713a97e14 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:09.454303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:09.490406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:09.491324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:09.491619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:09.491902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] PATH_INFO: `/v3/users/3cb45eb7f35848ea86cffb221f64f1dd` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:09.493121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb45eb7f35848ea86cffb221f64f1dd)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:09.495076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:09.546868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb45eb7f35848ea86cffb221f64f1dd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.555521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb45eb7f35848ea86cffb221f64f1dd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.563365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb45eb7f35848ea86cffb221f64f1dd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.564035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb45eb7f35848ea86cffb221f64f1dd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.564704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb45eb7f35848ea86cffb221f64f1dd'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.582789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1298/2575] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:09 2022] DELETE /identity/v3/users/3cb45eb7f35848ea86cffb221f64f1dd => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:09.790667 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:09.792162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:09.812066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c838f3-1be2-48b5-9f9c-7a317c404232 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:09.881695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:09.882818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:09.883289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:09.883565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] PATH_INFO: `/v3/projects/ca6b6983ee7342f993e12bb5cd0e0afd` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:09.884810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca6b6983ee7342f993e12bb5cd0e0afd)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:09.886661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:09.933746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca6b6983ee7342f993e12bb5cd0e0afd', 'request_id': 'req-8a3efcec-a53b-4c20-931f-c08f1b3068db'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.943891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca6b6983ee7342f993e12bb5cd0e0afd', 'request_id': 'req-8a3efcec-a53b-4c20-931f-c08f1b3068db'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.950703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca6b6983ee7342f993e12bb5cd0e0afd', 'request_id': 'req-8a3efcec-a53b-4c20-931f-c08f1b3068db'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.962105 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] The token cache is being invalidate because project ca6b6983ee7342f993e12bb5cd0e0afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:09.962509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.963191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-8a3efcec-a53b-4c20-931f-c08f1b3068db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:09.964792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1278/2576] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:09 2022] DELETE /identity/v3/projects/ca6b6983ee7342f993e12bb5cd0e0afd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:13.916626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:13.918083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:14.001486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ac666-d866-4e4d-bd4f-3d7ad0e0c97b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:14.049367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:14.068447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:14.069112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:14.069395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:14.069676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:14.071493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:14.084329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:14.085921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:14.096834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1299/2577] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:19.958923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=77,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:24.092075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=11 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:24.323134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:24.323838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:24.323838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:24.340710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] MFA Rules not processed for user `7f65bea891fb461086d6a60177ff2d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:24.341889 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:24.395052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1279/2578] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:48:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:24.698218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:24.699696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:24.760129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97f44c4-7a19-47fc-82d9-83b9d43be2c5 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:24.763725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:24.764377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:24.764655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:24.764929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:24.766709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:24.776402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:24.777831 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:24.786346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1300/2579] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:34.388155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=25 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:34.782330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=26 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:45.133915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:45.143358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcbf50-edc4-4f1a-b786-f7b81b7d27e5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:45.146815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:45.147480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:45.147761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:45.148033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:45.149778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:45.216079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:45.217514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:45.227475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1280/2580] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:48.141404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.143854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.160386 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4fcb8-587c-45df-9dcf-5082640dbce1 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.165521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.166415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.166703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.166982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] PATH_INFO: `/v3/users/9abdce8004e24e81bd88bee757d8ec93` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.168297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abdce8004e24e81bd88bee757d8ec93)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.170692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.233635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abdce8004e24e81bd88bee757d8ec93'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.241619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abdce8004e24e81bd88bee757d8ec93'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.249469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abdce8004e24e81bd88bee757d8ec93'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.250504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abdce8004e24e81bd88bee757d8ec93'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.251121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-000496fe-0b77-4576-b876-3938ed2ae7a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abdce8004e24e81bd88bee757d8ec93'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.270626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1301/2581] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:48:48 2022] DELETE /identity/v3/users/9abdce8004e24e81bd88bee757d8ec93 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:48.715083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.725179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85450e56-186a-4a1e-a299-b0b68161fb44 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:48.786180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:48.787040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.787361 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.787631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] PATH_INFO: `/v3/projects/edde4704253a4439bd7ca50b26e24e40` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.788805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edde4704253a4439bd7ca50b26e24e40)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:48.790711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:48.836122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edde4704253a4439bd7ca50b26e24e40', 'request_id': 'req-535af107-a525-41e7-af53-214961c1df85'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.842902 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edde4704253a4439bd7ca50b26e24e40', 'request_id': 'req-535af107-a525-41e7-af53-214961c1df85'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.849546 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edde4704253a4439bd7ca50b26e24e40', 'request_id': 'req-535af107-a525-41e7-af53-214961c1df85'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.859815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] The token cache is being invalidate because project edde4704253a4439bd7ca50b26e24e40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:48:48.860099 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.860733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-535af107-a525-41e7-af53-214961c1df85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:48:48.862874 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1281/2582] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:48:48 2022] DELETE /identity/v3/projects/edde4704253a4439bd7ca50b26e24e40 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:48:48.868932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.869225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.869508 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.886947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:48.888182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.947355 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1302/2583] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:48.955010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:48.955293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:48.955581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:48.975687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:48.976890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:48.985530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1282/2584] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:48:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:48.990259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:48.999902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7196aa-af3e-48d0-b8a8-616ef5244eb5 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.004442 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.005125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.005433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.005718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.006448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:48:49.006755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.008387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.019712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1303/2585] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:48:49.023840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.032845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa41158-fe10-416d-9446-c20ec3723c67 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.037956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.038636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.038918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.039191 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.039897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.041772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.055747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:49.077462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1283/2586] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:49.081461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.091403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922329e-aeee-4724-a2ae-b6f03d4c3d01 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.147363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.148032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.148321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.148601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.149287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.151235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.157735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:48:49.179263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1304/2587] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:48:49.181530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.192117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6813bb-4d33-4d07-ae36-bb235c5d7ca2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.196742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.197408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.197694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.197971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.198700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.200335 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.210973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1284/2588] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:49.215051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.225948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a687ae-f501-467c-98c4-44a4f9ac54d7 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.230649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1452b66a-3633-44e2-a68e-cf9a46409cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.231460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452b66a-3633-44e2-a68e-cf9a46409cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.231750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452b66a-3633-44e2-a68e-cf9a46409cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.232030 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1452b66a-3633-44e2-a68e-cf9a46409cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.232759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452b66a-3633-44e2-a68e-cf9a46409cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.234373 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1452b66a-3633-44e2-a68e-cf9a46409cf1 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.244550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1305/2589] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:48:49.248715 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.258778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c901be18-fd42-47e9-aeb5-36619dac947d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.263657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.264637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.264959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.265274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] PATH_INFO: `/v3/projects/f7c692366d8d456ba98167614a0921f0/users/063272f0c2054a53b831b35abf405b20/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.282516 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c692366d8d456ba98167614a0921f0, user_id=063272f0c2054a53b831b35abf405b20, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.285941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.299568 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1285/2590] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:48:49 2022] PUT /identity/v3/projects/f7c692366d8d456ba98167614a0921f0/users/063272f0c2054a53b831b35abf405b20/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:48:49.304681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94cdb0-6806-43fc-b18e-709ac91e99fd None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.304975 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94cdb0-6806-43fc-b18e-709ac91e99fd None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.305256 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94cdb0-6806-43fc-b18e-709ac91e99fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.323002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1e94cdb0-6806-43fc-b18e-709ac91e99fd None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:49.324257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1e94cdb0-6806-43fc-b18e-709ac91e99fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:49.383174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1306/2591] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:49.392506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:49.394677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:49.450851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea7995-41f5-49bf-848b-37806d6cc17d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:49.454581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:49.455258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:49.456029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:49.456308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:49.465653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:49.467086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:49.477204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1286/2592] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:54.322533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fabf6-c9c5-41e2-bace-b0d5f284bf8f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:54.323096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fabf6-c9c5-41e2-bace-b0d5f284bf8f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:54.323096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47fabf6-c9c5-41e2-bace-b0d5f284bf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:54.358732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c47fabf6-c9c5-41e2-bace-b0d5f284bf8f None None] MFA Rules not processed for user `063272f0c2054a53b831b35abf405b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:54.360322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c47fabf6-c9c5-41e2-bace-b0d5f284bf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:54.460779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1307/2593] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:48:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:48:54.467959 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:48:54.469698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:54.546011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac792e2-15e5-46ab-bebf-698d227fecb8 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:48:54.553610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:48:54.554269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:54.554562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:54.554822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:54.564295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:48:54.566498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:48:54.577346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1287/2594] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:48:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:48:55.910235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22560b90-44ef-4172-bc25-5c975b0d98ab None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:48:55.910770 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22560b90-44ef-4172-bc25-5c975b0d98ab None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:48:55.910939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22560b90-44ef-4172-bc25-5c975b0d98ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:48:55.929093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-22560b90-44ef-4172-bc25-5c975b0d98ab None None] MFA Rules not processed for user `063272f0c2054a53b831b35abf405b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:48:55.930378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-22560b90-44ef-4172-bc25-5c975b0d98ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:48:55.937273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1308/2595] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:48:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:04.573076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:05.923173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=3,keystone:SELECT=69 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:41.250189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:41.251616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:41.319358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f09b5-53f8-4701-b7ff-29ecb012d806 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:41.324188 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:41.324824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:41.325106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:41.325409 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:41.327101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:41.338832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:41.340443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:41.349974 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1288/2596] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:49:51.346179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=32 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:49:53.482222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:53.482708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:53.483004 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:53.502558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:49:53.503771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:53.559037 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1309/2597] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:49:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:49:53.604142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:53.605925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:53.675284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533245df-2799-42cb-8baf-cd14cbfba171 service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:53.680387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:53.681686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:53.681977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:53.682257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:53.690892 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:53.692547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:53.700608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1289/2598] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:49:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:49:59.813511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:49:59.816989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:59.832883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa36ffb0-3ba7-4bd7-92b5-4fb15ece0336 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:49:59.839295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:49:59.840565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:49:59.841928 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:49:59.842304 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:49:59.844265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:49:59.915733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:49:59.917171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7970a377-f08c-4f8e-8308-06e4e33a877d service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:49:59.928320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1310/2599] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:49:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:03.697353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=32 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:05.987969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:05.998215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b099e3-98bc-413b-b0f2-e17ba86f3f89 service swift] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:06.002441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:06.003020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:06.003333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:06.003646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:06.006440 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:06.019233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:06.020657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb3e11-1cf9-4934-b9ed-51fc33055c0e service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:06.030075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1290/2600] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:08.403262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e53e8f-6f6d-4241-ac46-b3760d70d051 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.404320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e53e8f-6f6d-4241-ac46-b3760d70d051 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.404620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e53e8f-6f6d-4241-ac46-b3760d70d051 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.406025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1311/2601] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:50:08.412434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69c5ff-295c-431c-9139-5c956ccf3c72 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.412881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69c5ff-295c-431c-9139-5c956ccf3c72 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.412983 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed69c5ff-295c-431c-9139-5c956ccf3c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.430897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ed69c5ff-295c-431c-9139-5c956ccf3c72 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:50:08.432154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ed69c5ff-295c-431c-9139-5c956ccf3c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:08.442028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1291/2602] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:50:08.453194 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e53e8f-6f6d-4241-ac46-b3760d70d051 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:08.455446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-12e53e8f-6f6d-4241-ac46-b3760d70d051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:08.530259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e53e8f-6f6d-4241-ac46-b3760d70d051 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:08.534678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665ddb9-f2f3-4b90-a8ca-1967c3705fe6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:08.535493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665ddb9-f2f3-4b90-a8ca-1967c3705fe6 service glance] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.535779 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665ddb9-f2f3-4b90-a8ca-1967c3705fe6 service glance] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.536056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665ddb9-f2f3-4b90-a8ca-1967c3705fe6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.547297 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665ddb9-f2f3-4b90-a8ca-1967c3705fe6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:08.549019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0665ddb9-f2f3-4b90-a8ca-1967c3705fe6 service glance] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:08.565505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1312/2603] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:08.636443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cab4-04e7-4dce-a153-266d0f3ca97a None None] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.636838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cab4-04e7-4dce-a153-266d0f3ca97a None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.637291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60cab4-04e7-4dce-a153-266d0f3ca97a None None] PATH_INFO: `/` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.639284 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1292/2604] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:50:08.651502 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b636383-696e-468c-be34-1cd5e09359f2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.651881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b636383-696e-468c-be34-1cd5e09359f2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.652223 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b636383-696e-468c-be34-1cd5e09359f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.672234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1b636383-696e-468c-be34-1cd5e09359f2 None None] MFA Rules not processed for user `95d010330c9c44ddb09dbd8c501f3087`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:50:08.673924 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1b636383-696e-468c-be34-1cd5e09359f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:08.681357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1313/2605] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:50:08.695310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60cab4-04e7-4dce-a153-266d0f3ca97a None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:08.711598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60cab4-04e7-4dce-a153-266d0f3ca97a None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:08.715683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82cd293-b6d3-4385-9c4e-36efd3a7302d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:08.716327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cd293-b6d3-4385-9c4e-36efd3a7302d service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.716658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cd293-b6d3-4385-9c4e-36efd3a7302d service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.716880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82cd293-b6d3-4385-9c4e-36efd3a7302d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.724624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cd293-b6d3-4385-9c4e-36efd3a7302d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:08.726063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82cd293-b6d3-4385-9c4e-36efd3a7302d service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:08.734502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1293/2606] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:50:08.843997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e26c8-32d7-4be9-85f3-dadb59023f43 None None] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.844427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e26c8-32d7-4be9-85f3-dadb59023f43 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.844801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e26c8-32d7-4be9-85f3-dadb59023f43 None None] PATH_INFO: `/` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.846984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1314/2607] 10.222.0.43 () {54 vars in 921 bytes} [Fri Jun 24 08:50:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 24 08:50:08.852867 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1af0-032b-41b7-a7c6-2fc790a401a7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.853156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1af0-032b-41b7-a7c6-2fc790a401a7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.853434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1af0-032b-41b7-a7c6-2fc790a401a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.872778 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-021c1af0-032b-41b7-a7c6-2fc790a401a7 None None] MFA Rules not processed for user `b502d4020b494807b09e803df0b2a301`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:50:08.874258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-021c1af0-032b-41b7-a7c6-2fc790a401a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:08.884595 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1294/2608] 10.222.0.43 () {58 vars in 1018 bytes} [Fri Jun 24 08:50:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:50:08.895903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e26c8-32d7-4be9-85f3-dadb59023f43 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:08.907308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e26c8-32d7-4be9-85f3-dadb59023f43 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:08.910827 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:08.911498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:08.911999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:08.912277 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:08.922732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:08.924593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:08.936287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1315/2609] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:10.720308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:10.720810 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:10.720911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:10.739328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:50:10.740645 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:10.748421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1295/2610] 10.222.0.43 () {58 vars in 1009 bytes} [Fri Jun 24 08:50:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:50:10.760605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:10.771789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c23b68a-ee12-445e-9dfd-fb961003dc7b service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:10.776654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:10.777300 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:10.777582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:10.777855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:10.786171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:10.787838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:10.796246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1316/2611] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:50:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:50:13.178740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:13.181490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:13.202172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d941e-7cb0-4b58-9b05-36a6141be514 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:13.209725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:13.210914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:13.211497 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:13.212387 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:13.214755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:13.228937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:13.231018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:13.245436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1296/2612] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:13.308391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:13.310301 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:13.342801 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518e816b-71fc-49a8-9f20-57b78c087031 service swift] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:13.362331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:13.363544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:13.363931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:13.364313 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] PATH_INFO: `/v3/users/7eb223c95f794ebd8d34f3a8627dfd89` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:13.365652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb223c95f794ebd8d34f3a8627dfd89)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:13.367811 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:13.374564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:13.433171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb223c95f794ebd8d34f3a8627dfd89'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.442434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb223c95f794ebd8d34f3a8627dfd89'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.452561 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb223c95f794ebd8d34f3a8627dfd89'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.453523 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb223c95f794ebd8d34f3a8627dfd89'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.454322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb223c95f794ebd8d34f3a8627dfd89'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.477814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1317/2613] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:50:13 2022] DELETE /identity/v3/users/7eb223c95f794ebd8d34f3a8627dfd89 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:50:13.668851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:13.684184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367150f9-e796-4beb-a8e5-43cb13c70b73 service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:13.753622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:13.754475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:13.754798 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:13.755069 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] PATH_INFO: `/v3/projects/d567a9e7ba344ca4ac65b276b57995d1` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:13.756322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d567a9e7ba344ca4ac65b276b57995d1)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:13.758490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:13.811530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd567a9e7ba344ca4ac65b276b57995d1', 'request_id': 'req-5a115c36-8f85-462c-b79b-92e70a94b188'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.820642 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd567a9e7ba344ca4ac65b276b57995d1', 'request_id': 'req-5a115c36-8f85-462c-b79b-92e70a94b188'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.828217 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd567a9e7ba344ca4ac65b276b57995d1', 'request_id': 'req-5a115c36-8f85-462c-b79b-92e70a94b188'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.839392 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] The token cache is being invalidate because project d567a9e7ba344ca4ac65b276b57995d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:50:13.839684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.840341 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:13.841927 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1297/2614] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:50:13 2022] DELETE /identity/v3/projects/d567a9e7ba344ca4ac65b276b57995d1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:50:13.846706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:13.848555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:13.862470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cfbc0-4d0f-4b74-98b0-20598d1ba4a3 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:13.930922 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:13.931837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:13.932124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:13.932403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] PATH_INFO: `/v3/users/81ff325bd75b4e478efcd9beda30eb31` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:13.933852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ff325bd75b4e478efcd9beda30eb31)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:13.935830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:13.988212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ff325bd75b4e478efcd9beda30eb31'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.000006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ff325bd75b4e478efcd9beda30eb31'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.013179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ff325bd75b4e478efcd9beda30eb31'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.014224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ff325bd75b4e478efcd9beda30eb31'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.015126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-a86abcd4-6962-4973-a6ad-e39952332b5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ff325bd75b4e478efcd9beda30eb31'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.036730 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1318/2615] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:50:13 2022] DELETE /identity/v3/users/81ff325bd75b4e478efcd9beda30eb31 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:50:14.252848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:14.254417 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:14.274225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115c36-8f85-462c-b79b-92e70a94b188 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:14.304466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:14.338170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:14.339116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:14.339506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:14.339915 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] PATH_INFO: `/v3/projects/64e08dd6d4d14f5981d81fad90d7bd34` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:14.341210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64e08dd6d4d14f5981d81fad90d7bd34)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:14.343240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:14.393681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64e08dd6d4d14f5981d81fad90d7bd34', 'request_id': 'req-bde9d1aa-a998-45b2-ae93-cef88178e0d3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.404305 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64e08dd6d4d14f5981d81fad90d7bd34', 'request_id': 'req-bde9d1aa-a998-45b2-ae93-cef88178e0d3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.413468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64e08dd6d4d14f5981d81fad90d7bd34', 'request_id': 'req-bde9d1aa-a998-45b2-ae93-cef88178e0d3'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.425428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] The token cache is being invalidate because project 64e08dd6d4d14f5981d81fad90d7bd34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:50:14.425709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.426393 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:50:14.427937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1298/2616] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:50:14 2022] DELETE /identity/v3/projects/64e08dd6d4d14f5981d81fad90d7bd34 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:50:24.033014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=76,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:24.421279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=37,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:31.016745 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:31.017698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:31.017803 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:31.034971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] MFA Rules not processed for user `5aa970dc0f084d6f9c46cb132964c02e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:50:31.036214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:31.089877 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1319/2617] 10.222.0.43 () {58 vars in 967 bytes} [Fri Jun 24 08:50:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:50:31.102795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:31.104205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:31.171693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d1aa-a998-45b2-ae93-cef88178e0d3 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:31.177095 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda72625-d708-4dc7-aae6-36ab40407bcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69edde6d461545069a9e98c661e5453e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:31.177749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72625-d708-4dc7-aae6-36ab40407bcd service swift] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:31.178033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72625-d708-4dc7-aae6-36ab40407bcd service swift] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:31.178311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda72625-d708-4dc7-aae6-36ab40407bcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:31.186456 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72625-d708-4dc7-aae6-36ab40407bcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:31.188081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda72625-d708-4dc7-aae6-36ab40407bcd service swift] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:31.196634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1299/2618] 10.222.0.43 () {60 vars in 1384 bytes} [Fri Jun 24 08:50:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 24 08:50:41.083423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=25 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:41.193248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=32 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:50:44.461875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:50:44.463315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:44.539320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea1e26c-6d69-4147-9047-d768a9916055 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:50:44.542868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:50:44.543548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:44.543830 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:44.544104 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:44.545835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:44.605395 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:50:44.606820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:50:44.615676 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1320/2619] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:50:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:50:54.572167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:50:54.572755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:50:54.572755 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:50:54.589602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] MFA Rules not processed for user `7f65bea891fb461086d6a60177ff2d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:50:54.590828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:50:54.598111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1300/2620] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:50:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:50:54.611574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=57 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:04.584869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:11.256564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:11.258858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:11.316464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a1e7f-c6ca-43f7-a2f2-857275cc308c service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:11.320202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:11.320849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:11.321133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:11.321411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:11.323177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:11.383318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:11.384728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:11.394578 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1321/2621] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:11 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:21.389847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=50 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:37.206598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:37.207992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:37.219570 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b44e1ea-87bd-4cbc-b777-0b2e94247f41 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:37.223668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:37.224299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:37.224578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:37.224848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:37.226521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:37.295403 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:37.296850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:37.307956 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1301/2622] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:38.486995 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:38.495689 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4951deab-b205-407a-9b02-b99c050cf495 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:38.499274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:38.499906 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:38.500184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:38.500454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:38.502162 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:38.515645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:38.517050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:38.528410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1322/2623] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:44.827421 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:44.836057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66556fff-132a-42e5-878c-d1aa711ab752 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:44.839818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:44.840460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:44.840739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:44.841012 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:44.842765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:44.904400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:44.905834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:44.915835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1302/2624] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:47.734783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:47.736767 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:47.751541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beaf4d29-1f84-45f5-8c86-22b11af91d0e service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:47.756339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:47.757298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:47.757655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:47.757946 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] PATH_INFO: `/v3/users/12f0ce0a293f4114a90098985e5f21ee` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:47.773274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f0ce0a293f4114a90098985e5f21ee)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:47.775237 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:47.820140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f0ce0a293f4114a90098985e5f21ee'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:47.827140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f0ce0a293f4114a90098985e5f21ee'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:47.834598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f0ce0a293f4114a90098985e5f21ee'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:47.835271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f0ce0a293f4114a90098985e5f21ee'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:47.835872 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4bddda2-37bc-4bb1-8b94-726713f893fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f0ce0a293f4114a90098985e5f21ee'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:47.853404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1323/2625] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:51:47 2022] DELETE /identity/v3/users/12f0ce0a293f4114a90098985e5f21ee => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:51:48.025604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.036057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c9116-a9e7-417d-a08f-d4123e7cb43e service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.092053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.092907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.093187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.093462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] PATH_INFO: `/v3/projects/150883a8e55c459a9f41fc48f8f81b0c` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.103483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=150883a8e55c459a9f41fc48f8f81b0c)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.105382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.116521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:48.150878 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '150883a8e55c459a9f41fc48f8f81b0c', 'request_id': 'req-18e0196b-138a-46df-80e1-463ee3b37347'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:48.158172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '150883a8e55c459a9f41fc48f8f81b0c', 'request_id': 'req-18e0196b-138a-46df-80e1-463ee3b37347'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:48.164912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '150883a8e55c459a9f41fc48f8f81b0c', 'request_id': 'req-18e0196b-138a-46df-80e1-463ee3b37347'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:48.180240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] The token cache is being invalidate because project 150883a8e55c459a9f41fc48f8f81b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:51:48.180525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:48.181194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-18e0196b-138a-46df-80e1-463ee3b37347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:51:48.182720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1303/2626] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:51:48 2022] DELETE /identity/v3/projects/150883a8e55c459a9f41fc48f8f81b0c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:51:48.187835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.188126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.188405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.207314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:48.208587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:48.266173 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1324/2627] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:48.274398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.274684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.274963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.292716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:48.294792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:48.305799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1304/2628] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:48.309911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.319898 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2acf6d7-affa-424c-a7d6-a047a4a1c4aa None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.324335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.324982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.325262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.325539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.326262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:51:48.326556 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.328171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.339960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1325/2629] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:51:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:51:48.344049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.353278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21223d08-3b93-42d1-a9cc-eaedafb04e1c None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.358399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.359109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.359413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.359690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.360383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.362288 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.390110 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1305/2630] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:51:48 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:51:48.394717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.404563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9625bdb4-9cff-40a5-b0d7-f10852fcdefe admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.437825 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:48.463741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.464324 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.464608 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.464887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.465563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.467498 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.485504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1326/2631] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:51:48 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:51:48.490049 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.500373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1565c2-701f-48e1-ae25-a7d09984d360 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.505581 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.506216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.506500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.506776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.507541 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.509139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.518885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1306/2632] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:51:48.523632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.533339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87373217-74d6-43d2-a46d-c91e786aa3a0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.537915 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae4daa-e3cc-4184-a109-c78e06fc2b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.538565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4daa-e3cc-4184-a109-c78e06fc2b97 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.538849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4daa-e3cc-4184-a109-c78e06fc2b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.539129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae4daa-e3cc-4184-a109-c78e06fc2b97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.539887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4daa-e3cc-4184-a109-c78e06fc2b97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.541495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae4daa-e3cc-4184-a109-c78e06fc2b97 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.551821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1327/2633] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:51:48.555932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.566818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4cef6-ac2b-4b2d-a761-0909a15ce3c8 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.571649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.572523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.572804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.573079 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] PATH_INFO: `/v3/projects/0ae612a1712a404681313b8758eb4f2d/users/8d3b1f091b4644d3b15b66eee01271c6/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.587450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ae612a1712a404681313b8758eb4f2d, user_id=8d3b1f091b4644d3b15b66eee01271c6, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.590102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.602888 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1307/2634] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:51:48 2022] PUT /identity/v3/projects/0ae612a1712a404681313b8758eb4f2d/users/8d3b1f091b4644d3b15b66eee01271c6/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:51:48.608076 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03dbc4-2d4e-4df3-bf41-920975bc0f5f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.608367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03dbc4-2d4e-4df3-bf41-920975bc0f5f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.608649 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b03dbc4-2d4e-4df3-bf41-920975bc0f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.626317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9b03dbc4-2d4e-4df3-bf41-920975bc0f5f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:48.627544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9b03dbc4-2d4e-4df3-bf41-920975bc0f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:48.684799 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1328/2635] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:51:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:48.695183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:48.696882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:48.751663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e25c19-bdba-4abb-b4af-38cf679df73e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:48.755793 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:48.756430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:48.756713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:48.756988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:48.766216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:48.767675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:48.776929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1308/2636] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:52.642619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf8ab-4535-46b9-b699-c43d53081bf3 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:52.642619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf8ab-4535-46b9-b699-c43d53081bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:52.643287 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437bf8ab-4535-46b9-b699-c43d53081bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:52.673150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-437bf8ab-4535-46b9-b699-c43d53081bf3 None None] MFA Rules not processed for user `8d3b1f091b4644d3b15b66eee01271c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:52.674325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-437bf8ab-4535-46b9-b699-c43d53081bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:52.747094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1329/2637] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:51:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:51:52.762181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:51:52.763663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:52.811170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:51:52.843425 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91c16c2-6359-4e58-9e6a-a4b9935bfb67 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:51:52.848947 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:51:52.849756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:52.850045 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:52.850325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:52.859163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:51:52.861007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:51:52.870648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1309/2638] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:51:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:51:54.126912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:51:54.127440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:51:54.127539 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:51:54.146494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] MFA Rules not processed for user `8d3b1f091b4644d3b15b66eee01271c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:51:54.147752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:51:54.155244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1330/2639] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:51:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:52:02.866203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=16 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:04.140425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=80,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:52:24.844707 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:52:24.846186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:24.858705 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864ab5d-f731-4fdd-ba6e-e957e5dc4b6b service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:52:24.862297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:52:24.862939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:52:24.863238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:52:24.863535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:52:24.865266 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:52:24.874891 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:52:24.876391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:52:24.884809 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1310/2640] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:52:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:52:34.881275 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=6 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:02.540856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:02.542514 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:02.563186 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee29a9d-1ebb-4b4a-a964-c327efc80f4d None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:02.570036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:02.571086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:02.571593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:02.572070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:02.574072 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:02.587703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:02.589490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:02.600997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1331/2641] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:05.252710 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:05.268148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951a75-1f3f-466f-9d16-67ecd1279f7e service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:05.273862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:05.274607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:05.274946 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:05.275365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:05.277158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:05.291432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:05.293015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:05.305625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1311/2642] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:12.596660 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:14.935166 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:14.945140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6172fb-88d3-4dad-a105-36f327528c58 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:14.948700 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:14.949331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:14.949611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:14.949884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:14.951631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:14.963422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:14.964997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:14.975525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1332/2643] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:15.300529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=6 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:20.867005 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:20.869814 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:20.883681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496673c4-7726-468a-8dfe-030a7c45a4c9 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:20.888532 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:20.889471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:20.889941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:20.890220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] PATH_INFO: `/v3/users/7f65bea891fb461086d6a60177ff2d65` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:20.891491 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f65bea891fb461086d6a60177ff2d65)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:20.894127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:20.945752 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f65bea891fb461086d6a60177ff2d65'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:20.953076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f65bea891fb461086d6a60177ff2d65'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:20.965115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f65bea891fb461086d6a60177ff2d65'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:20.972348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f65bea891fb461086d6a60177ff2d65'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:20.973009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f65bea891fb461086d6a60177ff2d65'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:20.978908 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1312/2644] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:53:20 2022] DELETE /identity/v3/users/7f65bea891fb461086d6a60177ff2d65 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:21.158221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:21.171310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45ec4a-9aa8-422a-ad44-64a0955667d9 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:21.229372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:21.230269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:21.230557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:21.230865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] PATH_INFO: `/v3/projects/891980c3542d4223a15461bb7c5dcff9` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:21.232083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] RBAC: Authorizing `identity:delete_project(project_id=891980c3542d4223a15461bb7c5dcff9)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:21.233974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:21.279812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891980c3542d4223a15461bb7c5dcff9', 'request_id': 'req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.285119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891980c3542d4223a15461bb7c5dcff9', 'request_id': 'req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.291348 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891980c3542d4223a15461bb7c5dcff9', 'request_id': 'req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.303954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] The token cache is being invalidate because project 891980c3542d4223a15461bb7c5dcff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:53:21.304241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.304875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.306469 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1333/2645] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:53:21 2022] DELETE /identity/v3/projects/891980c3542d4223a15461bb7c5dcff9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:21.310476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:21.312069 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:21.335243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf8486-3fb5-4904-871c-c9c3fb6fec86 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:21.384109 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:21.384958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:21.385241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:21.385514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] PATH_INFO: `/v3/users/8ee691ec36424bacadf4b0d26dc1edf7` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:21.386706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ee691ec36424bacadf4b0d26dc1edf7)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:21.388661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:21.438471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ee691ec36424bacadf4b0d26dc1edf7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.445944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ee691ec36424bacadf4b0d26dc1edf7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.458506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ee691ec36424bacadf4b0d26dc1edf7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.460500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:21.467775 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ee691ec36424bacadf4b0d26dc1edf7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.468453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-69a08da4-f88c-4754-a68a-2d93eb47c1ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ee691ec36424bacadf4b0d26dc1edf7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.475364 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1313/2646] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:53:21 2022] DELETE /identity/v3/users/8ee691ec36424bacadf4b0d26dc1edf7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:21.642847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:21.644454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:21.659094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f99270-8bb0-4a65-b9d8-9a2dcb1a196e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:21.729579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:21.730454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:21.730739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:21.731015 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] PATH_INFO: `/v3/projects/a5bc8829106841d6accb393f68d1f594` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:21.732293 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5bc8829106841d6accb393f68d1f594)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:21.734263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:21.755943 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:21.785562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5bc8829106841d6accb393f68d1f594', 'request_id': 'req-58713985-a266-4f17-b9e0-c3df46d10908'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.791737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5bc8829106841d6accb393f68d1f594', 'request_id': 'req-58713985-a266-4f17-b9e0-c3df46d10908'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.798411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5bc8829106841d6accb393f68d1f594', 'request_id': 'req-58713985-a266-4f17-b9e0-c3df46d10908'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.810828 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] The token cache is being invalidate because project a5bc8829106841d6accb393f68d1f594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:53:21.811113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.811755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:53:21.813602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1334/2647] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:53:21 2022] DELETE /identity/v3/projects/a5bc8829106841d6accb393f68d1f594 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:53:31.471789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:31.807879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:32.244039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:32.244330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:32.244609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:32.264229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] MFA Rules not processed for user `8d3b1f091b4644d3b15b66eee01271c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:32.265608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:32.319416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1314/2648] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:53:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:32.620812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:32.622232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:32.678150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58713985-a266-4f17-b9e0-c3df46d10908 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:32.681630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:32.682263 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:32.682544 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:32.682817 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:32.684563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:32.694351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:32.695793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bda1cc-4b94-4a97-adca-8a2ae2a6a778 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:32.703639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1335/2649] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:33.037375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:33.039630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:33.123176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea7fd76-aba2-4135-a1a8-079d6fc29d34 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:33.130599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:33.131384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:33.131772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:33.132209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:33.143965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:33.156116 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:33.157761 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:33.168818 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1315/2650] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:37.350759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:37.351319 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:37.351839 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:37.371706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] MFA Rules not processed for user `063272f0c2054a53b831b35abf405b20`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:53:37.372914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:37.427318 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1336/2651] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:53:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:53:37.727601 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:37.737462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9cc53-0bb2-4416-94bb-847e9145e229 service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:37.741027 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:37.741660 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:37.741944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:37.742221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:37.744138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:37.755163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:37.756820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:37.765268 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1316/2652] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:41.915822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:53:41.917332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:41.990425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a8a00-0250-48a9-b568-8969ee1b99cf None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:53:41.995856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:53:41.996496 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:53:41.996776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:53:41.997050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:53:41.998763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:53:42.011073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:53:42.012655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:53:42.022921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1337/2653] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:53:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:53:47.761457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=63 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:53:52.018628 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=85 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:54:45.227153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:54:45.285637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e652d453-7544-4dc6-ab5f-19eb4d3fed07 service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:54:45.290622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:54:45.291279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:54:45.291563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:54:45.291834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:54:45.293564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:54:45.325741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:54:45.327422 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c37f6e1-ffa5-4404-b3a3-ca6509926599 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:54:45.337563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1317/2654] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:54:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:54:55.333433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=36 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:04.310177 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:04.311677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:04.416893 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2012cf6e-31c3-4cd1-b844-2959dbb33cfd service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:04.423412 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:04.424659 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:04.425339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:04.425339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:04.427366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:04.440797 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:04.442650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:04.455640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1338/2655] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:55:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:55:14.448852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=40 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:20.213080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:20.213609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:20.213712 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:20.230724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] MFA Rules not processed for user `8d3b1f091b4644d3b15b66eee01271c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:20.231935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:20.239370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1318/2656] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:30.225981 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:38.047605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:38.067756 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff343c-53c8-4a2d-813a-9bdb899ef8d8 service nova] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:38.071316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:38.071957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:38.072241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:38.072518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:38.074265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:38.156849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:38.158772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:38.178360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1339/2657] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:55:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:55:41.400727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:41.423189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e4811-f233-4849-aeef-b1081405cfc2 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:41.429590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:41.430869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:41.431151 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:41.431995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:41.435193 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:41.452400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:41.454100 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:41.469641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:41.471821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:41.477843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1319/2658] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:55:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:55:41.489268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d72a80-1b45-45f0-b176-1ea82c020c34 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:41.494155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:41.495391 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:41.495684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:41.495965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] PATH_INFO: `/v3/users/063272f0c2054a53b831b35abf405b20` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:41.497171 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] RBAC: Authorizing `identity:delete_user(user_id=063272f0c2054a53b831b35abf405b20)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:41.499358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:41.547754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '063272f0c2054a53b831b35abf405b20'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.554929 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '063272f0c2054a53b831b35abf405b20'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.562204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '063272f0c2054a53b831b35abf405b20'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.562896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '063272f0c2054a53b831b35abf405b20'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.563528 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-e4b8770b-b555-4332-8d71-9c0a51e3007f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '063272f0c2054a53b831b35abf405b20'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.581423 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1340/2659] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:55:41 2022] DELETE /identity/v3/users/063272f0c2054a53b831b35abf405b20 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:41.752856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:41.765220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad8278-032f-4293-897c-33071011edc6 service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:41.822706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:41.823599 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:41.823882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:41.824158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] PATH_INFO: `/v3/projects/f7c692366d8d456ba98167614a0921f0` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:41.825338 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c692366d8d456ba98167614a0921f0)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:41.827269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:41.872400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c692366d8d456ba98167614a0921f0', 'request_id': 'req-399829f0-116a-41bb-a0c1-1d080a6cf750'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.880592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c692366d8d456ba98167614a0921f0', 'request_id': 'req-399829f0-116a-41bb-a0c1-1d080a6cf750'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.888089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c692366d8d456ba98167614a0921f0', 'request_id': 'req-399829f0-116a-41bb-a0c1-1d080a6cf750'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.899600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] The token cache is being invalidate because project f7c692366d8d456ba98167614a0921f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:55:41.899886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.900702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-399829f0-116a-41bb-a0c1-1d080a6cf750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:41.902228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1320/2660] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:55:41 2022] DELETE /identity/v3/projects/f7c692366d8d456ba98167614a0921f0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:41.907107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:41.907428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:41.907714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:41.944883 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:41.965149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:41.966360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:42.024246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1341/2661] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:42.032115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.032406 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.032693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.050941 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:42.052167 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:42.060329 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1321/2662] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:42.064789 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.074345 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b207db-3aaf-4d26-8df9-9dff6a40c9eb None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.078881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.079564 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.080401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.080687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.081416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:55:42.081726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.083341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.094621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1342/2663] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:55:42.098613 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.107819 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f9737-9d24-46a4-a034-4ed8228e9406 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.112452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.113094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.113383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.113661 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.114346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.116280 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.141220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1322/2664] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/projects => generated 357 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:55:42.145847 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.155732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca5ec09-c258-487b-b817-7081f4042b35 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.214739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.215462 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.215750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.216042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.216735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.218663 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.237932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1343/2665] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/users => generated 279 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:55:42.242015 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.252668 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719ac9e5-c12e-4036-8d13-8c1e57dd21a0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.257283 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.257938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.258221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.258498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.259241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.260841 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.271489 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1323/2666] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:55:42.275152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.285161 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53bd16-8c62-4c9a-b68f-8a5042ac598f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.289834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cd522-a63b-4e17-a617-74cf6887e115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.290518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd522-a63b-4e17-a617-74cf6887e115 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.290808 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd522-a63b-4e17-a617-74cf6887e115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.291088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cd522-a63b-4e17-a617-74cf6887e115 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.291874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cd522-a63b-4e17-a617-74cf6887e115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.293457 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64cd522-a63b-4e17-a617-74cf6887e115 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.304693 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1344/2667] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:55:42.308681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.319072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a00d1d-27c4-4443-92d4-1c8bbca45e7a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.323683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.324547 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.324830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.325103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] PATH_INFO: `/v3/projects/76d650613409457294cc32f0d0c80afc/users/ea668cd197a2496d934cc7fcde13aa7f/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.339700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d650613409457294cc32f0d0c80afc, user_id=ea668cd197a2496d934cc7fcde13aa7f, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.342386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.354977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1324/2668] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:55:42 2022] PUT /identity/v3/projects/76d650613409457294cc32f0d0c80afc/users/ea668cd197a2496d934cc7fcde13aa7f/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:42.366140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.366435 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.366713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.394338 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] MFA Rules not processed for user `ea668cd197a2496d934cc7fcde13aa7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:42.395584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:42.455006 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:42.466048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1345/2669] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:42.470616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.483693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:42.535683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a6b23-1d39-4677-b45d-ef3ca71c5f16 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.540226 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.540890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.541177 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.541465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.542163 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.544136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.570166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1325/2670] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:55:42.574312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.585038 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46be12f-62bf-4160-b7fb-abb02b568707 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.640349 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.641008 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.641288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.641562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.642240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.644413 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.664775 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1346/2671] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:55:42.668733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.680052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6b7e-3c3a-4933-947b-e468d5af3a29 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.685085 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.685738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.686017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.686290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.687011 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.688655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.699068 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1326/2672] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:55:42.703405 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.713455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2aaae-81be-4cc7-ba4d-90872f3fa785 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.719080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.720019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.720796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.721073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] PATH_INFO: `/v3/projects/ce8350fa25014c9ab12307030231d2c2/users/e044a46a59994f3080842c75f829b302/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.737059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8350fa25014c9ab12307030231d2c2, user_id=e044a46a59994f3080842c75f829b302, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.739843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.754239 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1347/2673] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:55:42 2022] PUT /identity/v3/projects/ce8350fa25014c9ab12307030231d2c2/users/e044a46a59994f3080842c75f829b302/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:42.758390 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.768444 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b8cd3-743d-47d0-962f-7d9d84d9c5a4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.824186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5e613a-15af-4d0b-8a2e-7241e06c87ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.824851 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e613a-15af-4d0b-8a2e-7241e06c87ae admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.825134 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e613a-15af-4d0b-8a2e-7241e06c87ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.825414 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5e613a-15af-4d0b-8a2e-7241e06c87ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.826137 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e613a-15af-4d0b-8a2e-7241e06c87ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.827767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5e613a-15af-4d0b-8a2e-7241e06c87ae admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.837448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1327/2674] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:55:42.841551 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.852512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106fffd5-08b5-4197-86d0-00297d5c033c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:42.857330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:42.858214 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.858499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.858778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] PATH_INFO: `/v3/projects/ce8350fa25014c9ab12307030231d2c2/users/e044a46a59994f3080842c75f829b302/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.860786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce8350fa25014c9ab12307030231d2c2, user_id=e044a46a59994f3080842c75f829b302, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:42.864129 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:42.874396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1348/2675] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:55:42 2022] PUT /identity/v3/projects/ce8350fa25014c9ab12307030231d2c2/users/e044a46a59994f3080842c75f829b302/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:42.884647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:42.884925 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:42.885201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:42.911656 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] MFA Rules not processed for user `e044a46a59994f3080842c75f829b302`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:42.912895 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:42.980382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1328/2676] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:42 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:42.992010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:42.993709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:43.052181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6eed09-690c-4d39-88ca-26ba908bfd5c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:43.055860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:43.056507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:43.056792 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:43.057069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:43.066089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:43.067636 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:43.077231 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1349/2677] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:55:42 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:55:43.149221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:43.159382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16173787-4181-4c24-baec-eba7b53d80d3 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:43.162939 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e7deb9-fcba-4552-9495-a90890931a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:43.163612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7deb9-fcba-4552-9495-a90890931a37 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:43.163905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7deb9-fcba-4552-9495-a90890931a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:43.164176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e7deb9-fcba-4552-9495-a90890931a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:43.169898 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:43.225080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7deb9-fcba-4552-9495-a90890931a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:43.226529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e7deb9-fcba-4552-9495-a90890931a37 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:43.236248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1329/2678] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:55:43 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:55:53.072836 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:53.231703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=24 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:57.995560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:58.055960 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca204d-4b04-4003-9171-c69f069c5f96 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:58.060364 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:58.061229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:58.061510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:58.061785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] PATH_INFO: `/v3/users/ea668cd197a2496d934cc7fcde13aa7f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:58.062967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea668cd197a2496d934cc7fcde13aa7f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:58.064961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:58.109579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea668cd197a2496d934cc7fcde13aa7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.117066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea668cd197a2496d934cc7fcde13aa7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.124000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea668cd197a2496d934cc7fcde13aa7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.124616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea668cd197a2496d934cc7fcde13aa7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.125224 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea668cd197a2496d934cc7fcde13aa7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.142440 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1350/2679] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:55:57 2022] DELETE /identity/v3/users/ea668cd197a2496d934cc7fcde13aa7f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:58.146882 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:58.147420 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:58.147529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:58.163769 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:58.164954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:58.222378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1330/2680] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:55:58.232102 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:58.233467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:58.301207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f9ad3-9d9d-41cc-aa5a-29806564f095 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:58.304832 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:58.305483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:58.305771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:58.306049 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:58.314886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:58.316939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:58.326185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1351/2681] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:55:58.591635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:58.593204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:58.611136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30e05ff-b9f4-4883-893d-fac6756216d1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:58.616372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:58.617241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:58.617522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:58.617869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] PATH_INFO: `/v3/projects/76d650613409457294cc32f0d0c80afc` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:58.618972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d650613409457294cc32f0d0c80afc)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:58.620932 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:58.666029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d650613409457294cc32f0d0c80afc', 'request_id': 'req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.672584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d650613409457294cc32f0d0c80afc', 'request_id': 'req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.679623 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d650613409457294cc32f0d0c80afc', 'request_id': 'req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.690101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] The token cache is being invalidate because project 76d650613409457294cc32f0d0c80afc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:55:58.690385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.691009 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.692575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1331/2682] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:55:58 2022] DELETE /identity/v3/projects/76d650613409457294cc32f0d0c80afc => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:58.696619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:58.706249 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539c89ac-06ec-46fd-88bd-45114e6b468c service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:58.714639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:58.764642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:58.765532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:58.765870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:58.766159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] PATH_INFO: `/v3/users/e044a46a59994f3080842c75f829b302` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:58.767353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e044a46a59994f3080842c75f829b302)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:58.769785 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:58.818097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e044a46a59994f3080842c75f829b302'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.826002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e044a46a59994f3080842c75f829b302'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.834807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e044a46a59994f3080842c75f829b302'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.835403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e044a46a59994f3080842c75f829b302'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.836133 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-9ee58ede-5a9d-4af1-b6fb-3aefebdb6468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e044a46a59994f3080842c75f829b302'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:58.853433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1352/2683] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:55:58 2022] DELETE /identity/v3/users/e044a46a59994f3080842c75f829b302 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:59.121584 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.123119 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:59.136559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1a2d7-eabc-4bbc-b47c-ddb9704d0226 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.194502 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.195430 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.195720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.195997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] PATH_INFO: `/v3/projects/ce8350fa25014c9ab12307030231d2c2` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.197183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce8350fa25014c9ab12307030231d2c2)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.199103 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.230164 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:59.247472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce8350fa25014c9ab12307030231d2c2', 'request_id': 'req-32efe55c-1c60-4bb1-b4b0-358e82973590'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:59.255062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce8350fa25014c9ab12307030231d2c2', 'request_id': 'req-32efe55c-1c60-4bb1-b4b0-358e82973590'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:59.261332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce8350fa25014c9ab12307030231d2c2', 'request_id': 'req-32efe55c-1c60-4bb1-b4b0-358e82973590'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:59.272182 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] The token cache is being invalidate because project ce8350fa25014c9ab12307030231d2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:55:59.272474 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:59.273113 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-32efe55c-1c60-4bb1-b4b0-358e82973590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:55:59.274639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1332/2684] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:55:59 2022] DELETE /identity/v3/projects/ce8350fa25014c9ab12307030231d2c2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:55:59.279571 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.280073 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.280172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.297951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:59.299144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:59.357257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1353/2685] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:55:59.366628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.366890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.367178 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.385397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:59.386587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:59.394504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1333/2686] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:59.398557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.408146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbb7656-172c-440d-ae0f-d1d20e6cd596 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.413191 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.413841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.414125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.414404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.415127 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:55:59.415465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.417070 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.428914 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1354/2687] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:55:59.433013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.442133 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dadfc5-8881-41e5-85af-117a3ac90114 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.446724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.447398 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.447684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.447961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.448650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.450549 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.476311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1334/2688] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:55:59.480409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.490834 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cd611e-c22e-493d-abcb-82f7202a516e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.503056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:55:59.548931 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.549772 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.549905 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.550150 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.551666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.553563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.574393 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1355/2689] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:55:59.578622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.589069 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e63cbc-72ac-4e29-b729-a796745947e6 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.593766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.594419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.594701 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.594977 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.595764 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.597372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.607536 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1335/2690] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:55:59.611427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.621713 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc82339-5f92-43c7-8f46-2185e8353994 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.626172 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596231c2-683c-497a-9629-93a22318cafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.626814 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596231c2-683c-497a-9629-93a22318cafa admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.627094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596231c2-683c-497a-9629-93a22318cafa admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.627397 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596231c2-683c-497a-9629-93a22318cafa admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.628115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596231c2-683c-497a-9629-93a22318cafa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.630325 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596231c2-683c-497a-9629-93a22318cafa admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.639971 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1356/2691] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:55:59.643961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.653647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f465aa-7619-4c5d-9f62-159c5198e9e9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.658206 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.659071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.659395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.659675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] PATH_INFO: `/v3/projects/ff9928fdd64d4ab1ba576513a5af6243/users/d27ff24f5e0e4304ae65aabe9706d0ab/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.674744 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff9928fdd64d4ab1ba576513a5af6243, user_id=d27ff24f5e0e4304ae65aabe9706d0ab, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.678208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.692535 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1336/2692] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:55:59 2022] PUT /identity/v3/projects/ff9928fdd64d4ab1ba576513a5af6243/users/d27ff24f5e0e4304ae65aabe9706d0ab/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:55:59.702759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.703050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.703383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.730323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] MFA Rules not processed for user `d27ff24f5e0e4304ae65aabe9706d0ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:55:59.731572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:55:59.792740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1357/2693] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:55:59.797071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.857096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bce15b-c520-4568-8fda-7f696e3c9ee5 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.861649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.862295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.862579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.862856 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.864144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.866041 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.890368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1337/2694] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:55:59.894365 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.904932 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7852aa4-9f5e-4be0-9965-4d68c05c15f2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:55:59.960228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:55:59.960896 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:55:59.961182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:55:59.961458 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:55:59.962149 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:55:59.964269 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:55:59.983942 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1358/2695] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:55:59 2022] POST /identity/v3/users => generated 298 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:55:59.987914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:55:59.998909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c27341-9097-4ab8-a599-bf714a2eefa2 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:00.003496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:00.004179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.004461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.004750 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.005460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:00.007075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:00.018490 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1338/2696] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:00.022445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:00.032200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fc5a5d-57f7-409b-8b89-57ed06599523 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:00.037396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:00.038273 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.038583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.038866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] PATH_INFO: `/v3/projects/7d106c355650481eae9734d202f789c2/users/7052a6f9b59f4831859e3c3fa0775104/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.051503 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:00.054876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d106c355650481eae9734d202f789c2, user_id=7052a6f9b59f4831859e3c3fa0775104, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:00.057994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:00.071158 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1359/2697] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:00 2022] PUT /identity/v3/projects/7d106c355650481eae9734d202f789c2/users/7052a6f9b59f4831859e3c3fa0775104/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:00.075111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:00.085089 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f5c096-be7c-4311-9122-5a7407520431 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:00.104994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:00.142828 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e7fc6-321b-46ba-9b7d-ee6958fb1f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:00.143455 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e7fc6-321b-46ba-9b7d-ee6958fb1f74 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.143743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e7fc6-321b-46ba-9b7d-ee6958fb1f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.144018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e7fc6-321b-46ba-9b7d-ee6958fb1f74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.144766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e7fc6-321b-46ba-9b7d-ee6958fb1f74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:00.146363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e7fc6-321b-46ba-9b7d-ee6958fb1f74 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:00.155933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1339/2698] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:00.160533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:00.171084 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb29f767-86fc-4088-875a-c672af371367 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:00.176292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:00.177169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.177448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.177747 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] PATH_INFO: `/v3/projects/7d106c355650481eae9734d202f789c2/users/7052a6f9b59f4831859e3c3fa0775104/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.179740 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d106c355650481eae9734d202f789c2, user_id=7052a6f9b59f4831859e3c3fa0775104, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:00.182358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:00.192840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1360/2699] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:00 2022] PUT /identity/v3/projects/7d106c355650481eae9734d202f789c2/users/7052a6f9b59f4831859e3c3fa0775104/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:56:00.203235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.203723 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.203834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.230345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] MFA Rules not processed for user `7052a6f9b59f4831859e3c3fa0775104`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:00.231561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:00.296799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1340/2700] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:00.307752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:00.309424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:00.364782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51def4be-76f0-46c4-8e32-d68dd769be43 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:00.368357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:00.368984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.369267 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.369541 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.378289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:00.379773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:00.388947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1361/2701] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:00 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:00.485527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:00.495075 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85806afb-4196-4311-8ab4-5ece63a9779e None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:00.498607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108dfba1-1d5a-4b29-97d7-fc98d9b12053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:00.499272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dfba1-1d5a-4b29-97d7-fc98d9b12053 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:00.499562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dfba1-1d5a-4b29-97d7-fc98d9b12053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:00.499844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108dfba1-1d5a-4b29-97d7-fc98d9b12053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:00.555150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dfba1-1d5a-4b29-97d7-fc98d9b12053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:00.556625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108dfba1-1d5a-4b29-97d7-fc98d9b12053 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:00.565432 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1341/2702] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:03.103774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:03.169290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1729312-1ec2-47bf-a30a-a8e39e86da86 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:03.174071 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:03.174958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:03.175268 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:03.175553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] PATH_INFO: `/v3/users/d27ff24f5e0e4304ae65aabe9706d0ab` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:03.177553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27ff24f5e0e4304ae65aabe9706d0ab)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:03.179557 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:03.226511 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27ff24f5e0e4304ae65aabe9706d0ab'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.234858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27ff24f5e0e4304ae65aabe9706d0ab'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.241776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27ff24f5e0e4304ae65aabe9706d0ab'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.242619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27ff24f5e0e4304ae65aabe9706d0ab'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.243358 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27ff24f5e0e4304ae65aabe9706d0ab'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.257761 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:03.262311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1362/2703] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:03 2022] DELETE /identity/v3/users/d27ff24f5e0e4304ae65aabe9706d0ab => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:03.267021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:03.267539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:03.267650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:03.284887 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:03.286694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:03.325855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=100 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:03.345159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1342/2704] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:56:03.355199 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:03.356616 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:03.413085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a7225-2b25-4c05-9292-7c50347892d7 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:03.417406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:03.418053 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:03.418335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:03.418626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:03.427907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:03.429327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:03.438705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1363/2705] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:03.700104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:03.701616 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:03.714399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13255dc1-85e9-4c36-9f2f-59a94b66bfa7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:03.718971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:03.719879 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:03.720159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:03.720433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] PATH_INFO: `/v3/projects/ff9928fdd64d4ab1ba576513a5af6243` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:03.721617 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff9928fdd64d4ab1ba576513a5af6243)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:03.723696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:03.768824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff9928fdd64d4ab1ba576513a5af6243', 'request_id': 'req-ec6da525-f6ba-4aee-b018-af00caa34b7f'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.776813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff9928fdd64d4ab1ba576513a5af6243', 'request_id': 'req-ec6da525-f6ba-4aee-b018-af00caa34b7f'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.782994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff9928fdd64d4ab1ba576513a5af6243', 'request_id': 'req-ec6da525-f6ba-4aee-b018-af00caa34b7f'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.793360 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] The token cache is being invalidate because project ff9928fdd64d4ab1ba576513a5af6243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:03.793649 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.794287 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.795849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1343/2706] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:03 2022] DELETE /identity/v3/projects/ff9928fdd64d4ab1ba576513a5af6243 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:03.799774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:03.809445 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa44e36-bf57-4278-b6bd-48a77f410891 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:03.865389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:03.866271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:03.866560 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:03.866842 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] PATH_INFO: `/v3/users/7052a6f9b59f4831859e3c3fa0775104` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:03.868064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7052a6f9b59f4831859e3c3fa0775104)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:03.870055 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:03.915840 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7052a6f9b59f4831859e3c3fa0775104'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.922673 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7052a6f9b59f4831859e3c3fa0775104'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.930574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7052a6f9b59f4831859e3c3fa0775104'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.931262 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7052a6f9b59f4831859e3c3fa0775104'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.931861 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-1ddf1b07-091f-4b73-af44-22bf7e7f4ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7052a6f9b59f4831859e3c3fa0775104'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:03.949259 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1364/2707] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:03 2022] DELETE /identity/v3/users/7052a6f9b59f4831859e3c3fa0775104 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:04.223872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.225877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:04.241428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6da525-f6ba-4aee-b018-af00caa34b7f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.297210 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.298067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.298348 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.298620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] PATH_INFO: `/v3/projects/7d106c355650481eae9734d202f789c2` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.299805 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d106c355650481eae9734d202f789c2)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.302833 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.346566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d106c355650481eae9734d202f789c2', 'request_id': 'req-424e6c22-4105-4fa9-86c1-55b069a04c39'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:04.353454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d106c355650481eae9734d202f789c2', 'request_id': 'req-424e6c22-4105-4fa9-86c1-55b069a04c39'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:04.359418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d106c355650481eae9734d202f789c2', 'request_id': 'req-424e6c22-4105-4fa9-86c1-55b069a04c39'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:04.366608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:04.372062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] The token cache is being invalidate because project 7d106c355650481eae9734d202f789c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:04.372346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:04.373353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-424e6c22-4105-4fa9-86c1-55b069a04c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:04.375227 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1344/2708] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:04 2022] DELETE /identity/v3/projects/7d106c355650481eae9734d202f789c2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:04.380807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.381310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.381865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.399605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:04.400804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:04.417153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:04.461577 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1365/2709] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:56:04.470278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.470569 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.470848 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.490121 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:04.491289 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:04.499140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1345/2710] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:04.505264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.515078 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aa00cb-692d-4f15-b365-a5af8016dd62 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.519699 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.521048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.521332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.521612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.522363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:56:04.522669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.524312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.535838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1366/2711] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:56:04.540122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.552796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e5f301-5b7e-418c-9263-d507ed136971 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.557408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.558052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.558333 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.558603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.559347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.561231 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.586484 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1346/2712] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:56:04 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:56:04.590533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.600878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c65e1e-0d82-4537-9900-f1c726e443d5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.656644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.657305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.657591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.657868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.658548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.660487 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.680210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1367/2713] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:56:04 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:56:04.684225 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.695136 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b984bfe-aca0-4777-a9b3-ff22a792c430 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.699703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.700350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.700629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.700900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.701617 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.703200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.714381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1347/2714] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:04.718057 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.727889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2123b-6aae-4a31-802a-fa90264b0583 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.732418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9df223f-c2fe-4045-beab-d87931f3ce6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.733064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df223f-c2fe-4045-beab-d87931f3ce6d admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.733350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df223f-c2fe-4045-beab-d87931f3ce6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.733629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9df223f-c2fe-4045-beab-d87931f3ce6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.734351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df223f-c2fe-4045-beab-d87931f3ce6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.735963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9df223f-c2fe-4045-beab-d87931f3ce6d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.746362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1368/2715] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:04.750265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.760825 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba756f0-d8ad-4458-b031-f1113cb67ffa admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.765388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.766257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.766542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.766820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] PATH_INFO: `/v3/projects/4ee0e12c52ec41a9b05e006e168c1d5d/users/2439d9de40724970bf72113e46c19a7f/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.781327 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee0e12c52ec41a9b05e006e168c1d5d, user_id=2439d9de40724970bf72113e46c19a7f, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.784059 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:04.797741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1348/2716] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:04 2022] PUT /identity/v3/projects/4ee0e12c52ec41a9b05e006e168c1d5d/users/2439d9de40724970bf72113e46c19a7f/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:04.808593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.808911 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.809192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.838046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] MFA Rules not processed for user `2439d9de40724970bf72113e46c19a7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:04.839305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:04.902820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1369/2717] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:04.907204 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:04.967560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda8457-ce23-4bd2-a940-3e1fea59eb30 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:04.972043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:04.972685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:04.972966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:04.973241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:04.973913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:04.975918 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.000171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1349/2718] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:56:04 2022] POST /identity/v3/projects => generated 383 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:56:05.004244 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.016019 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3e20d-37d1-4d28-acf8-0888d254569c None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.027935 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:05.072647 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.073308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.073590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.073870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.074548 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.076477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.095229 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1370/2719] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:56:05 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 08:56:05.099716 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.109740 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00ae843-048f-432a-a096-a1ecbbc2794b admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.114852 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.115534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.115811 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.116083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.116800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.118384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.128711 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1350/2720] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 08:56:05.133549 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.145310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6467af11-c4ef-4e24-a7d5-338093698e7b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.151081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.152002 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.152285 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.152558 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] PATH_INFO: `/v3/projects/1a6162926a8f4b3c9be6a8ace9f8a59b/users/7dfb7611a5ab43e5a94b4ef919f52749/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.168059 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6162926a8f4b3c9be6a8ace9f8a59b, user_id=7dfb7611a5ab43e5a94b4ef919f52749, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.170705 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.186220 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1371/2721] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:05 2022] PUT /identity/v3/projects/1a6162926a8f4b3c9be6a8ace9f8a59b/users/7dfb7611a5ab43e5a94b4ef919f52749/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:05.191143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.202162 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b4b6b-0c17-4b24-acd0-4e99129e1e39 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.256840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a74ab9-fb20-46e9-8495-07b6016db5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.257550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a74ab9-fb20-46e9-8495-07b6016db5fe admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.257836 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a74ab9-fb20-46e9-8495-07b6016db5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.258223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a74ab9-fb20-46e9-8495-07b6016db5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.258944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a74ab9-fb20-46e9-8495-07b6016db5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.260610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a74ab9-fb20-46e9-8495-07b6016db5fe admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.271363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1351/2722] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:05.275383 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.285685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cecfc86-93c4-41c7-97eb-b817bb568bf0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.290782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.291701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.291987 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.292265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] PATH_INFO: `/v3/projects/1a6162926a8f4b3c9be6a8ace9f8a59b/users/7dfb7611a5ab43e5a94b4ef919f52749/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.294212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6162926a8f4b3c9be6a8ace9f8a59b, user_id=7dfb7611a5ab43e5a94b4ef919f52749, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.296900 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.308146 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1372/2723] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:05 2022] PUT /identity/v3/projects/1a6162926a8f4b3c9be6a8ace9f8a59b/users/7dfb7611a5ab43e5a94b4ef919f52749/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:56:05.319092 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.319593 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.319720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.326347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:05.346449 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] MFA Rules not processed for user `7dfb7611a5ab43e5a94b4ef919f52749`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:05.347676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:05.414370 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1352/2724] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:05.425433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.427991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:05.484639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde1fe5-598e-4ae7-8e41-bdce69e5377c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.488266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.488952 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.489222 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.489501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.498468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.499921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.510184 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1373/2725] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:05.577646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.587800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195f141-f9c7-4d2a-a7a6-689535620355 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.591376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f34e0d-6307-454c-a3c5-0cba53f377a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.592004 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f34e0d-6307-454c-a3c5-0cba53f377a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.592290 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f34e0d-6307-454c-a3c5-0cba53f377a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.592560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f34e0d-6307-454c-a3c5-0cba53f377a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.647794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f34e0d-6307-454c-a3c5-0cba53f377a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.649221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f34e0d-6307-454c-a3c5-0cba53f377a0 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.657865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1353/2726] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:05.906543 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:05.967933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea337c53-61d0-4708-a5aa-1799a697948a service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:05.972401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:05.973759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:05.974043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:05.974322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] PATH_INFO: `/v3/users/2439d9de40724970bf72113e46c19a7f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:05.975505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2439d9de40724970bf72113e46c19a7f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:05.977407 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:05.990999 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:06.025010 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2439d9de40724970bf72113e46c19a7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.031897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2439d9de40724970bf72113e46c19a7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.040200 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2439d9de40724970bf72113e46c19a7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.040845 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2439d9de40724970bf72113e46c19a7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.041476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2439d9de40724970bf72113e46c19a7f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.059838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1374/2727] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:05 2022] DELETE /identity/v3/users/2439d9de40724970bf72113e46c19a7f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:06.064295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:06.064586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:06.064869 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:06.081324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:06.082534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:06.140219 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1354/2728] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:06.149696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:06.151080 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:06.206917 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4677cc-f606-4909-a7b7-1a6a643b7a37 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:06.211042 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:06.211715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:06.212000 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:06.212278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:06.221167 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:06.222582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:06.232464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1375/2729] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:06.490937 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:06.492470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:06.505591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea462a06-9202-49f4-8903-ed2d675860aa None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:06.514207 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:06.515065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:06.515375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:06.515680 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] PATH_INFO: `/v3/projects/4ee0e12c52ec41a9b05e006e168c1d5d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:06.516885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee0e12c52ec41a9b05e006e168c1d5d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:06.518840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:06.546641 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:06.574142 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee0e12c52ec41a9b05e006e168c1d5d', 'request_id': 'req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.581258 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee0e12c52ec41a9b05e006e168c1d5d', 'request_id': 'req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.588057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee0e12c52ec41a9b05e006e168c1d5d', 'request_id': 'req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.598346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] The token cache is being invalidate because project 4ee0e12c52ec41a9b05e006e168c1d5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:06.598637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.599326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.600871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1355/2730] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:06 2022] DELETE /identity/v3/projects/4ee0e12c52ec41a9b05e006e168c1d5d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:06.604976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:06.614602 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae66f4c-e5c4-42ff-851b-ca33a13dccb4 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:06.669454 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:06.670317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:06.670600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:06.670879 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] PATH_INFO: `/v3/users/7dfb7611a5ab43e5a94b4ef919f52749` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:06.672091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfb7611a5ab43e5a94b4ef919f52749)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:06.673993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:06.710347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:06.719812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfb7611a5ab43e5a94b4ef919f52749'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.726620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfb7611a5ab43e5a94b4ef919f52749'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.733083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfb7611a5ab43e5a94b4ef919f52749'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.733753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfb7611a5ab43e5a94b4ef919f52749'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.734389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-793543d9-f87f-4df4-a08b-fc6561685bba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfb7611a5ab43e5a94b4ef919f52749'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:06.751185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1376/2731] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:06 2022] DELETE /identity/v3/users/7dfb7611a5ab43e5a94b4ef919f52749 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:07.033655 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.035080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:07.048319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3e2505-1931-4e1e-a97e-e8d197ee8fe7 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.104664 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.105450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.105758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.106033 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] PATH_INFO: `/v3/projects/1a6162926a8f4b3c9be6a8ace9f8a59b` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.107190 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6162926a8f4b3c9be6a8ace9f8a59b)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.109122 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.153988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6162926a8f4b3c9be6a8ace9f8a59b', 'request_id': 'req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:07.160586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6162926a8f4b3c9be6a8ace9f8a59b', 'request_id': 'req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:07.166844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6162926a8f4b3c9be6a8ace9f8a59b', 'request_id': 'req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:07.179428 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] The token cache is being invalidate because project 1a6162926a8f4b3c9be6a8ace9f8a59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:07.179748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:07.180606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-be4aed09-bdfa-4e66-a866-92bfc4e92b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:07.182208 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1356/2732] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:07 2022] DELETE /identity/v3/projects/1a6162926a8f4b3c9be6a8ace9f8a59b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:07.186996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.187320 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.187603 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.205882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:07.207083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:07.265051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1377/2733] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:07.272907 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.273203 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.273488 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.293807 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:07.294996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:07.303673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1357/2734] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:07.307661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.317156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779da429-d634-44bb-918d-6fa37e8029ca None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.321623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.322264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.322545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.322816 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.323572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:56:07.323874 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.325478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.336666 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1378/2735] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:56:07.340729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.350346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da4fee-1cf0-4336-8e5f-2a3ebeb84a32 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.355018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.355638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.355922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.356198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.356880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.358794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.389096 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1358/2736] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:56:07 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:56:07.389215 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.399434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83d4a8-81ba-42a2-ba30-a0eb935bc9bd admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.454226 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.454888 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.455174 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.455484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.456179 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.458112 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.477646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1379/2737] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:56:07 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:56:07.481673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.492334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c8bf6f-8da8-4eaf-ae76-91382cd8ef05 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.501337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.502000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.502285 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.502562 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.503433 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.505056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.515620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1359/2738] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:07.519614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.529531 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7234ce9-ee1a-4a06-971d-f15ba295e8a6 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.534069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ffe19f-60bb-408b-b26c-3c2ab5106b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.534715 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffe19f-60bb-408b-b26c-3c2ab5106b96 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.534996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffe19f-60bb-408b-b26c-3c2ab5106b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.535305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ffe19f-60bb-408b-b26c-3c2ab5106b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.536032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffe19f-60bb-408b-b26c-3c2ab5106b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.538121 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ffe19f-60bb-408b-b26c-3c2ab5106b96 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.547786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1380/2739] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:07.551855 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.557533 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:07.563462 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223951-6a98-41d1-8651-0f89a7e02da7 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.569411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.570279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.570555 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.570823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] PATH_INFO: `/v3/projects/cbaa22b1bf454a879f1200f04d6835ec/users/457ca17414a841b8aacf98a445ce1c38/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.585013 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbaa22b1bf454a879f1200f04d6835ec, user_id=457ca17414a841b8aacf98a445ce1c38, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.587687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.600265 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1360/2740] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:07 2022] PUT /identity/v3/projects/cbaa22b1bf454a879f1200f04d6835ec/users/457ca17414a841b8aacf98a445ce1c38/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:07.610706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.611183 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.611357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.638643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] MFA Rules not processed for user `457ca17414a841b8aacf98a445ce1c38`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:07.639852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:07.648280 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:07.705317 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1381/2741] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:56:07.716384 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:07.718130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:07.773173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad3e942-5938-4f7d-83f0-256355c63430 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:07.776749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329a084e-4186-4622-a17b-1267865f3c07 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:07.777382 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a084e-4186-4622-a17b-1267865f3c07 service glance] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:07.777663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a084e-4186-4622-a17b-1267865f3c07 service glance] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:07.777934 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329a084e-4186-4622-a17b-1267865f3c07 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:07.785992 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a084e-4186-4622-a17b-1267865f3c07 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:07.788052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329a084e-4186-4622-a17b-1267865f3c07 service glance] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:07.796673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1361/2742] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:17.702461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=23 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:17.792695 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:30.522550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:30.585321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2a3c9-729b-41fc-8f85-6b6509c4a200 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:30.589835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:30.590704 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:30.590989 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:30.591298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] PATH_INFO: `/v3/users/457ca17414a841b8aacf98a445ce1c38` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:30.592473 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] RBAC: Authorizing `identity:delete_user(user_id=457ca17414a841b8aacf98a445ce1c38)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:30.594378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:30.639867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '457ca17414a841b8aacf98a445ce1c38'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:30.646590 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '457ca17414a841b8aacf98a445ce1c38'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:30.653951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '457ca17414a841b8aacf98a445ce1c38'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:30.654579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '457ca17414a841b8aacf98a445ce1c38'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:30.655252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '457ca17414a841b8aacf98a445ce1c38'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:30.672305 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1382/2743] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:56:30 2022] DELETE /identity/v3/users/457ca17414a841b8aacf98a445ce1c38 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:30.676980 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:30.677466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:30.677574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:30.693786 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:30.694985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:30.751356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1362/2744] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:30.761258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:30.762640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:30.819436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7848ac49-15bb-4dd8-b6d0-d07470979ffc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:30.823655 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088ab14-a6eb-4134-8ac8-df36bda61571 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:30.824295 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088ab14-a6eb-4134-8ac8-df36bda61571 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:30.824579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088ab14-a6eb-4134-8ac8-df36bda61571 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:30.824859 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088ab14-a6eb-4134-8ac8-df36bda61571 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:30.833759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088ab14-a6eb-4134-8ac8-df36bda61571 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:30.835266 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088ab14-a6eb-4134-8ac8-df36bda61571 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:30.844598 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1383/2745] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:31.105839 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.107328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:31.121378 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1671ae-c590-4945-bc6c-6a05d23bf5d6 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.126161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.127022 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.127325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.127604 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] PATH_INFO: `/v3/projects/cbaa22b1bf454a879f1200f04d6835ec` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.129503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbaa22b1bf454a879f1200f04d6835ec)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.131698 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.178997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbaa22b1bf454a879f1200f04d6835ec', 'request_id': 'req-dce70dd1-5090-4610-8b43-b68fcc731c99'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:31.186339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbaa22b1bf454a879f1200f04d6835ec', 'request_id': 'req-dce70dd1-5090-4610-8b43-b68fcc731c99'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:31.192579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbaa22b1bf454a879f1200f04d6835ec', 'request_id': 'req-dce70dd1-5090-4610-8b43-b68fcc731c99'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:31.203434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] The token cache is being invalidate because project cbaa22b1bf454a879f1200f04d6835ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:56:31.203770 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:31.204363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-dce70dd1-5090-4610-8b43-b68fcc731c99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:56:31.205876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1363/2746] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:56:31 2022] DELETE /identity/v3/projects/cbaa22b1bf454a879f1200f04d6835ec => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:56:31.210823 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.211119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.211455 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.229043 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:31.230234 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:31.246595 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:31.290144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1384/2747] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:31.347235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.347540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.347820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.365812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:31.367144 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:31.374933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1364/2748] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:56:31.379140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.389392 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba1462c-0143-4c99-8e06-b16912f7fb89 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.393927 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.394583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.394871 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.395153 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.396416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:56:31.396735 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.398526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.410600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1385/2749] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:56:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:56:31.414727 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.423885 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06655df5-4d4a-4329-b92d-daf62e10fba0 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.428725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.429359 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.429635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.429910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.430592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.432514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.457886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1365/2750] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:56:31 2022] POST /identity/v3/projects => generated 367 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:56:31.461654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.472933 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3e991f-a66a-4250-8193-920014489aa7 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.528731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.529379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.529661 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.529963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.530651 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.532581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.552137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1386/2751] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:56:31 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:56:31.556646 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.566689 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70865e43-4593-4735-90e0-a3d79a25e003 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.571345 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.572622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.572905 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.573184 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.573906 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.575561 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.585843 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1366/2752] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:31.589788 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.599804 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d18a7-b166-420d-a097-28499ddd1588 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.604488 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c77401-930d-40c5-ac79-9200bbb22c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.605141 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c77401-930d-40c5-ac79-9200bbb22c54 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.605431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c77401-930d-40c5-ac79-9200bbb22c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.605712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c77401-930d-40c5-ac79-9200bbb22c54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.606438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c77401-930d-40c5-ac79-9200bbb22c54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.608062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c77401-930d-40c5-ac79-9200bbb22c54 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.617793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1387/2753] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:56:31.623158 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.633071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3432c-70ba-404c-9077-2112eb3b99ac admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.638215 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.639088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.639402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.639681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] PATH_INFO: `/v3/projects/f040d44336f14e3b988fb4d7225f3028/users/4ee3e96d169e48458c7e3f09907488b9/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.653884 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f040d44336f14e3b988fb4d7225f3028, user_id=4ee3e96d169e48458c7e3f09907488b9, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.656565 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.668987 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1367/2754] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:56:31 2022] PUT /identity/v3/projects/f040d44336f14e3b988fb4d7225f3028/users/4ee3e96d169e48458c7e3f09907488b9/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:56:31.679014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.679410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.679698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.706265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] MFA Rules not processed for user `4ee3e96d169e48458c7e3f09907488b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:56:31.707518 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:31.775706 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1388/2755] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:56:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:56:31.786883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:31.788574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:31.811780 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:31.867349 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261b13a-2822-403b-8fee-bb7023d85a75 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:31.871551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ffea41aca5bd4fd49e2c5f74ac317814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:31.872205 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:31.872494 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:31.872772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:31.881194 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:31.882640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:31.892098 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1368/2756] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:31 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:41.772739 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:41.888010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=26 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:56:50.202853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:50.204371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:50.259334 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c986ab90-446b-447a-a1f7-8dee84a963c9 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:50.262808 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:50.263470 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:50.263755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:50.264032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:50.265732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:50.334181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:50.335654 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:50.347596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1389/2757] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:52.694758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:52.704453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4655b049-7dc2-4e6a-ad1b-1fc88ef2d7e1 service glance] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:52.708150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:52.708782 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:52.709066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:52.709342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:52.711076 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:52.769386 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:52.770820 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd377789-2c78-4dd7-9699-086a90157cc4 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:52.780135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1369/2758] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:56:59.240236 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:56:59.241652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:59.252819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ec7b2f-7c0b-4363-9fe2-b926fc682ca4 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:56:59.256330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:56:59.256962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:56:59.257243 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:56:59.257520 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:56:59.259202 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:56:59.272868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:56:59.274288 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:56:59.285388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1390/2759] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:56:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:57:02.775527 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=28 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:57:09.280476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=60 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:57:12.636774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9c81c-98b3-46b1-ad30-59e8836ac24b None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:12.637278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9c81c-98b3-46b1-ad30-59e8836ac24b None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:12.637401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd9c81c-98b3-46b1-ad30-59e8836ac24b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:12.654371 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-efd9c81c-98b3-46b1-ad30-59e8836ac24b None None] MFA Rules not processed for user `8d3b1f091b4644d3b15b66eee01271c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:12.655574 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-efd9c81c-98b3-46b1-ad30-59e8836ac24b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:12.662624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1370/2760] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:16.643278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:16.706328 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda90a37-39dd-4624-83e9-9540c488552a service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:16.711107 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:16.712017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:16.712302 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:16.712582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] PATH_INFO: `/v3/users/4ee3e96d169e48458c7e3f09907488b9` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:16.714281 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee3e96d169e48458c7e3f09907488b9)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:16.716245 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:16.764046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee3e96d169e48458c7e3f09907488b9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:16.771474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee3e96d169e48458c7e3f09907488b9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:16.779863 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee3e96d169e48458c7e3f09907488b9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:16.780499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee3e96d169e48458c7e3f09907488b9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:16.781119 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee3e96d169e48458c7e3f09907488b9'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:16.799919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1391/2761] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:57:16 2022] DELETE /identity/v3/users/4ee3e96d169e48458c7e3f09907488b9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:57:16.804401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:16.804864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:16.804961 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:16.822213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:16.823445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:16.882069 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1371/2762] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:16.891949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:16.893367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:16.949996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0f6a5f-5346-47b0-9dd8-6c5d73af6998 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:16.953667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afe191-f5ce-4903-9827-ee24679993f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:16.954314 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afe191-f5ce-4903-9827-ee24679993f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:16.954600 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afe191-f5ce-4903-9827-ee24679993f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:16.954881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afe191-f5ce-4903-9827-ee24679993f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:16.964493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afe191-f5ce-4903-9827-ee24679993f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:16.965913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afe191-f5ce-4903-9827-ee24679993f3 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:16.975467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1392/2763] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:57:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:57:17.230772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.232396 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:17.244979 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b548f3-de00-4310-81f5-82a09a0e60c8 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.249767 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.250792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.251071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.251383 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] PATH_INFO: `/v3/projects/f040d44336f14e3b988fb4d7225f3028` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.253281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f040d44336f14e3b988fb4d7225f3028)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.255301 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.300586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f040d44336f14e3b988fb4d7225f3028', 'request_id': 'req-21ee1955-df33-44e9-8e29-3227d3d2188e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:17.308216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f040d44336f14e3b988fb4d7225f3028', 'request_id': 'req-21ee1955-df33-44e9-8e29-3227d3d2188e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:17.314592 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f040d44336f14e3b988fb4d7225f3028', 'request_id': 'req-21ee1955-df33-44e9-8e29-3227d3d2188e'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:17.325235 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] The token cache is being invalidate because project f040d44336f14e3b988fb4d7225f3028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:57:17.325518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:17.326130 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-21ee1955-df33-44e9-8e29-3227d3d2188e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:57:17.327741 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1372/2764] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:57:17 2022] DELETE /identity/v3/projects/f040d44336f14e3b988fb4d7225f3028 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:57:17.332623 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.332944 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.333227 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.350333 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:17.351565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:17.368145 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:57:17.413248 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1393/2765] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:17.422148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.422435 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.422737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.441747 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:17.442929 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:17.451308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1373/2766] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:17.455474 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.464881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df81ff3-6553-48d2-8006-2ab6ad1fbf55 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.469344 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.469988 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.470272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.470752 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.471294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:57:17.471652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.473240 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.484546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1394/2767] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:57:17.488600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.498186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e515eeb-3e40-4460-9d88-ffbe155e5988 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.502835 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.503521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.503804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.504080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.504760 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.506653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.532521 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1374/2768] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:57:17 2022] POST /identity/v3/projects => generated 383 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:57:17.536601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.546481 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740d513-84ee-4e1f-bcf6-07c075c0bfa9 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.600675 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.601329 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.601611 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.601884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.602567 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.604515 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.623056 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1395/2769] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:57:17 2022] POST /identity/v3/users => generated 292 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:57:17.627259 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.637837 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6377e9ac-6d43-46a0-87c9-10c909d12841 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.642347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.642995 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.643309 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.643578 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.644299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.645910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.656071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1375/2770] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:57:17.659725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.669958 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73519d32-3c57-4b13-8f59-ed0f480ca1a3 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.674449 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5560888-849a-4387-a025-60a9ced73efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.675101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560888-849a-4387-a025-60a9ced73efb admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.675419 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560888-849a-4387-a025-60a9ced73efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.675701 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5560888-849a-4387-a025-60a9ced73efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.676677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5560888-849a-4387-a025-60a9ced73efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.678414 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5560888-849a-4387-a025-60a9ced73efb admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.688907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1396/2771] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:57:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:57:17.692823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.702450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b89d46-6979-4c8b-8d20-43a4d69d6c97 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.706962 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.707853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.708135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.708410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] PATH_INFO: `/v3/projects/6fe7f78cbcdb45c8946ed1fe9171e7b1/users/60dddf53ee2647d08d2c5f3b17f34e1a/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.722334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe7f78cbcdb45c8946ed1fe9171e7b1, user_id=60dddf53ee2647d08d2c5f3b17f34e1a, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.725029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.738146 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1376/2772] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:57:17 2022] PUT /identity/v3/projects/6fe7f78cbcdb45c8946ed1fe9171e7b1/users/60dddf53ee2647d08d2c5f3b17f34e1a/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:57:17.742796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8528d1-29f3-4549-838b-afe564320f30 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.743092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8528d1-29f3-4549-838b-afe564320f30 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.743417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8528d1-29f3-4549-838b-afe564320f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.761341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-da8528d1-29f3-4549-838b-afe564320f30 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:17.762526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-da8528d1-29f3-4549-838b-afe564320f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:17.822126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1397/2773] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:17.831724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:17.833408 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:17.848504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:57:17.894101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531d1dc-324c-488e-87e5-eba71468e8e4 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:17.898199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:17.898876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:17.899236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:17.899552 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:17.909368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:17.910864 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:17.921314 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1377/2774] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:57:18.751122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc292-4f6e-453e-b08a-b7dad90e8477 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:18.751711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc292-4f6e-453e-b08a-b7dad90e8477 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:18.751998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc292-4f6e-453e-b08a-b7dad90e8477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:18.783190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-358bc292-4f6e-453e-b08a-b7dad90e8477 None None] MFA Rules not processed for user `60dddf53ee2647d08d2c5f3b17f34e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:18.784894 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-358bc292-4f6e-453e-b08a-b7dad90e8477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:18.803783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:57:18.864807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1398/2775] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:18.875729 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:18.877324 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:18.938871 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52e897-5ca3-4346-ac08-bfd1a5a12a6f service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:18.942453 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:18.943102 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:18.943446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:18.943719 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:18.952520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:18.954030 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:18.965615 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1378/2776] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:57:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:57:20.317950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:20.318447 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:20.318641 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:20.336478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] MFA Rules not processed for user `60dddf53ee2647d08d2c5f3b17f34e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:57:20.337650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:20.344377 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1399/2777] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:57:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:57:24.466154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:24.467720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:24.469870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:57:24.471347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:24.481138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda759b-931d-464f-b1d5-cadafc15bb77 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:24.485055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:24.485709 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:24.485996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:24.486278 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:24.486387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6beb02-a04e-45ad-b8fa-09d7b2b44f15 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:57:24.488213 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:57:24.490411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d016e611-9c75-47af-b379-9a14e71cb91d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:57:24.491093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016e611-9c75-47af-b379-9a14e71cb91d service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:57:24.491416 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016e611-9c75-47af-b379-9a14e71cb91d service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:57:24.491822 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d016e611-9c75-47af-b379-9a14e71cb91d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:57:24.498573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:24.500086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:24.500448 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016e611-9c75-47af-b379-9a14e71cb91d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:57:24.501912 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d016e611-9c75-47af-b379-9a14e71cb91d service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:57:24.510017 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1379/2778] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:57:24.512681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1400/2779] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:57:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:57:34.504804 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=52 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:57:34.506620 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=28 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:58:22.600498 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:58:22.602086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:58:22.679507 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed95ba4-9537-46ac-82e8-637c2fa1809e service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:58:22.685652 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:58:22.686306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:58:22.686587 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:58:22.686862 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:58:22.688706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:58:22.701914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:58:22.703550 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:58:22.714399 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1380/2780] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:58:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:58:32.709830 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=34 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:58:33.351870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:58:33.352402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:58:33.352402 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:58:33.370643 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] MFA Rules not processed for user `60dddf53ee2647d08d2c5f3b17f34e1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:58:33.371873 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:58:33.379017 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1401/2781] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:58:33 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:58:33.934733 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:58:33.936279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:58:34.016725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0776205-707c-4783-a6b4-a6d0941b01a8 service nova] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:58:34.021658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:58:34.022297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:58:34.022579 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:58:34.022910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:58:34.025738 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:58:34.037823 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:58:34.040114 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:58:34.050233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1381/2782] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:58:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:58:43.364967 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:58:44.045872 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=32 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:36.077483 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:36.090265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df2b84-9cda-4ad6-8433-ec597db4239e None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:36.095439 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:36.096098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:36.096390 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:36.096669 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:36.098467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:36.110486 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:36.112115 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:36.124018 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1402/2783] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:59:36.137880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:36.149070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb66557-c42c-4061-96dc-acfd73b7342a service placement] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:36.154312 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:36.155332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:36.155718 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:36.156097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] PATH_INFO: `/v3/users/60dddf53ee2647d08d2c5f3b17f34e1a` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:36.157401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60dddf53ee2647d08d2c5f3b17f34e1a)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:36.159572 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:36.215563 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60dddf53ee2647d08d2c5f3b17f34e1a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.223093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60dddf53ee2647d08d2c5f3b17f34e1a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.236436 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60dddf53ee2647d08d2c5f3b17f34e1a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.243781 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60dddf53ee2647d08d2c5f3b17f34e1a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.244470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60dddf53ee2647d08d2c5f3b17f34e1a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.250325 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1382/2784] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:59:36 2022] DELETE /identity/v3/users/60dddf53ee2647d08d2c5f3b17f34e1a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:36.425380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:36.437363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a225df4-6e45-4e7c-a083-fc620b3fb7ba service placement] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:36.495718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:36.496582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:36.496865 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:36.497143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] PATH_INFO: `/v3/projects/6fe7f78cbcdb45c8946ed1fe9171e7b1` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:36.498312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe7f78cbcdb45c8946ed1fe9171e7b1)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:36.500260 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:36.545003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe7f78cbcdb45c8946ed1fe9171e7b1', 'request_id': 'req-22cdb7ac-4747-4189-a196-2ded8e0475f0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.550484 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe7f78cbcdb45c8946ed1fe9171e7b1', 'request_id': 'req-22cdb7ac-4747-4189-a196-2ded8e0475f0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.557033 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe7f78cbcdb45c8946ed1fe9171e7b1', 'request_id': 'req-22cdb7ac-4747-4189-a196-2ded8e0475f0'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.570330 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] The token cache is being invalidate because project 6fe7f78cbcdb45c8946ed1fe9171e7b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:59:36.570619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.571257 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:36.572796 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1403/2785] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:59:36 2022] DELETE /identity/v3/projects/6fe7f78cbcdb45c8946ed1fe9171e7b1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:43.491368 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:43.492834 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:43.549304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf137c3-22c8-46f7-a037-6a2926b00d86 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:43.596272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:43.596916 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:43.597199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:43.597478 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:43.599366 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:43.651923 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:43.675996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:43.677724 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:43.688304 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1383/2786] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:59:46.567124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:46.634069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:46.635677 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:46.649375 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cdb7ac-4747-4189-a196-2ded8e0475f0 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:46.653939 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:46.655003 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:46.655321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:46.655606 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] PATH_INFO: `/v3/users/8d3b1f091b4644d3b15b66eee01271c6` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:46.656771 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d3b1f091b4644d3b15b66eee01271c6)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:46.658671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:46.705716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d3b1f091b4644d3b15b66eee01271c6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:46.712614 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d3b1f091b4644d3b15b66eee01271c6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:46.719703 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d3b1f091b4644d3b15b66eee01271c6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:46.720315 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d3b1f091b4644d3b15b66eee01271c6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:46.720920 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-552f87f7-b74d-4e63-9fa2-42b6acc50c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d3b1f091b4644d3b15b66eee01271c6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:46.740211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1404/2787] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:59:46 2022] DELETE /identity/v3/users/8d3b1f091b4644d3b15b66eee01271c6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:46.913539 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:46.931247 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c46c63-be34-4fb3-bb0f-b279f390c4ac service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:46.988728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:46.989597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:46.989881 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:46.990159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] PATH_INFO: `/v3/projects/0ae612a1712a404681313b8758eb4f2d` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:46.991363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ae612a1712a404681313b8758eb4f2d)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:46.993293 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.038600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ae612a1712a404681313b8758eb4f2d', 'request_id': 'req-caedc976-4935-49be-ae1f-07ced28e1d99'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:47.046148 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ae612a1712a404681313b8758eb4f2d', 'request_id': 'req-caedc976-4935-49be-ae1f-07ced28e1d99'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:47.052838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ae612a1712a404681313b8758eb4f2d', 'request_id': 'req-caedc976-4935-49be-ae1f-07ced28e1d99'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:47.063115 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] The token cache is being invalidate because project 0ae612a1712a404681313b8758eb4f2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:59:47.063431 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:47.064066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-caedc976-4935-49be-ae1f-07ced28e1d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:47.065629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1384/2788] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:59:46 2022] DELETE /identity/v3/projects/0ae612a1712a404681313b8758eb4f2d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:47.070843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.071255 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.071652 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.091461 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:47.092580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:47.158083 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1405/2789] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:59:47.166606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.167135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.167746 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.188090 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:47.189286 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:47.197787 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1385/2790] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:59:47.201991 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.212553 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdc7544-af11-4f9b-9ab1-525574cc7664 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.217438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.218105 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.218387 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.218664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.219428 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:59:47.219738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.221327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.233357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1406/2791] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:59:47.237742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.250070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91ccd9-3226-4959-aa57-dc00b20a9848 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.254673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.255356 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.255653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.255922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.256637 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.258554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.295419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1386/2792] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:59:47 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:59:47.299773 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.310848 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaa0106-6c6d-4fd6-9605-c17b12296d22 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.368252 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.368838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.369120 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.369399 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.370126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.372106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.378688 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:47.394860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1407/2793] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:59:47 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:59:47.399088 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.409567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb09bb0-688e-42f2-bb81-98ecea8da685 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.414928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.415624 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.415909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.416186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.416913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.418545 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.428713 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1387/2794] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:59:47.433221 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.443599 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe0e98d-fdac-4102-a392-b2b6933c7a5b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.448323 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa4cfe9-e2f7-45ff-b91b-952e3a5088ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.448998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4cfe9-e2f7-45ff-b91b-952e3a5088ec admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.449284 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4cfe9-e2f7-45ff-b91b-952e3a5088ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.449563 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa4cfe9-e2f7-45ff-b91b-952e3a5088ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.450294 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4cfe9-e2f7-45ff-b91b-952e3a5088ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.451983 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa4cfe9-e2f7-45ff-b91b-952e3a5088ec admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.462296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1408/2795] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:59:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:59:47.466503 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.477077 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee00291-2a94-4135-97cc-fe837336865c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.481993 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.482877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.483161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.483471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] PATH_INFO: `/v3/projects/2eb292d467c14eafa6d20a66ad49f795/users/b465582691f842b2a9aa8cac0dbe6804/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.498057 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb292d467c14eafa6d20a66ad49f795, user_id=b465582691f842b2a9aa8cac0dbe6804, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.500840 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.503824 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:47.525514 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1388/2796] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:59:47 2022] PUT /identity/v3/projects/2eb292d467c14eafa6d20a66ad49f795/users/b465582691f842b2a9aa8cac0dbe6804/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:47.537734 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.538026 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.538310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.569403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] MFA Rules not processed for user `b465582691f842b2a9aa8cac0dbe6804`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:47.571169 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:47.638513 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1409/2797] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:59:47.650405 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:47.652129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:47.723970 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0794d5-c5ff-4c80-ba75-7e4b49b5acda admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:47.730080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7148ae-4be0-45c8-b436-4769c32894b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b502d4020b494807b09e803df0b2a301', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:47.730720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7148ae-4be0-45c8-b436-4769c32894b0 service nova] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:47.731006 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7148ae-4be0-45c8-b436-4769c32894b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:47.731322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7148ae-4be0-45c8-b436-4769c32894b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:47.738985 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7148ae-4be0-45c8-b436-4769c32894b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:47.740598 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7148ae-4be0-45c8-b436-4769c32894b0 service nova] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:47.749313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1389/2798] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:47 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:59:50.487058 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:50.548206 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b03e6-2d86-4cca-a2be-b1ed195bc32f None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:50.552913 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:50.553783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:50.554065 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:50.554363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] PATH_INFO: `/v3/users/b465582691f842b2a9aa8cac0dbe6804` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:50.555601 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b465582691f842b2a9aa8cac0dbe6804)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:50.557510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:50.604100 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b465582691f842b2a9aa8cac0dbe6804'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:50.611046 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b465582691f842b2a9aa8cac0dbe6804'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:50.618881 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b465582691f842b2a9aa8cac0dbe6804'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:50.619537 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b465582691f842b2a9aa8cac0dbe6804'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:50.620155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b465582691f842b2a9aa8cac0dbe6804'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:50.625495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:INSERT=4,keystone:SELECT=87,keystone:DELETE=9 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:50.644160 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1410/2799] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 08:59:50 2022] DELETE /identity/v3/users/b465582691f842b2a9aa8cac0dbe6804 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:50.648772 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:50.649244 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:50.649346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:50.666334 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:50.667607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:50.726609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1390/2800] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:59:50.736207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:50.737717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:50.796040 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d225a3-ecdc-4b8b-a86d-694f715aa8c6 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:50.799708 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:50.800357 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:50.800642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:50.800921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:50.810696 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:50.812143 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:50.822136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1411/2801] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:59:51.092560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.094468 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:51.111000 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027a26a5-3e2f-4ecd-af44-e78188eff5b7 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.116376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.117395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.117794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.118070 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] PATH_INFO: `/v3/projects/2eb292d467c14eafa6d20a66ad49f795` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.119452 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb292d467c14eafa6d20a66ad49f795)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.121808 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.138996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.140410 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:51.178326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb292d467c14eafa6d20a66ad49f795', 'request_id': 'req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:51.186968 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb292d467c14eafa6d20a66ad49f795', 'request_id': 'req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:51.193628 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb292d467c14eafa6d20a66ad49f795', 'request_id': 'req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:51.206321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] The token cache is being invalidate because project 2eb292d467c14eafa6d20a66ad49f795 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 08:59:51.206609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:51.207273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-493f0b00-25bb-44d9-9e7b-e9df5941b8c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 08:59:51.209124 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1391/2802] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 08:59:51 2022] DELETE /identity/v3/projects/2eb292d467c14eafa6d20a66ad49f795 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 08:59:51.214080 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5cbfc-0bd7-4b0b-96d1-e61be098c475 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.214365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5cbfc-0bd7-4b0b-96d1-e61be098c475 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.214639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5cbfc-0bd7-4b0b-96d1-e61be098c475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.218346 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845437b-07b6-4add-a97a-0ac25adb888b service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.221726 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:51.235730 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-73b5cbfc-0bd7-4b0b-96d1-e61be098c475 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:51.237567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-73b5cbfc-0bd7-4b0b-96d1-e61be098c475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:51.279631 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.280203 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.280485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.280758 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.282550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:51.294289 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.295963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.296799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1392/2803] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:59:51.305339 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.305631 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.305911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.307479 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1412/2804] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 08:59:51.324255 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:51.325363 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:51.333189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1393/2805] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 08:59:51.337464 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.348050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64bced-c840-441c-a06e-b16d4f322b4a service placement] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.353204 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.353821 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.354111 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.354393 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.355122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 08:59:51.355465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.357092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.370009 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1413/2806] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 08:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 08:59:51.374295 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.383858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c309c-1abc-489d-9115-40300dee0848 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.388606 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.389201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.389461 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.389813 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.390518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.392591 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.419381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1394/2807] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:59:51.423687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.434743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8059bd12-47e6-4068-99f3-edf40275ca5d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.440394 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:51.495418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.496152 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.496406 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.496685 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.497378 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.499339 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.519841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1414/2808] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:59:51.524416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.534991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440ae60-42e9-4c22-8703-9c82242bed9e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.540218 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.540873 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.541159 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.541483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.542209 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.543886 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.555505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1395/2809] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:59:51.559477 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.569424 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43c34d6-2b8e-49e9-8d3c-bdc7f6722e45 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.574093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ab0815-328d-4502-a974-9a54698d8b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.574743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0815-328d-4502-a974-9a54698d8b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.575025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0815-328d-4502-a974-9a54698d8b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.575342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ab0815-328d-4502-a974-9a54698d8b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.576069 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab0815-328d-4502-a974-9a54698d8b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.577642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ab0815-328d-4502-a974-9a54698d8b0c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.588754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1415/2810] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:59:51.592742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.603043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ff3555-d8bb-4ec9-a19f-315491d8f2e0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.607765 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.608647 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.608914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.609189 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] PATH_INFO: `/v3/projects/4e180bbd7b9c4e81a09db5a5e7fdc1bf/users/e9fd13fa406b4fa5bbf75d8af47e9c2e/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.623671 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e180bbd7b9c4e81a09db5a5e7fdc1bf, user_id=e9fd13fa406b4fa5bbf75d8af47e9c2e, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.626328 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.639086 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1396/2811] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:59:51 2022] PUT /identity/v3/projects/4e180bbd7b9c4e81a09db5a5e7fdc1bf/users/e9fd13fa406b4fa5bbf75d8af47e9c2e/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:51.649702 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.649997 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.650274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.678342 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] MFA Rules not processed for user `e9fd13fa406b4fa5bbf75d8af47e9c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:51.679581 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:51.744331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1416/2812] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:59:51.748971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.810756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ad628d-6f43-4cdc-83a7-70a9cea3984d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.815385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.816034 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.816315 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.816590 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.817281 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.819176 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.844117 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1397/2813] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:59:51.848371 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.864265 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b9cad-f097-4b2a-811e-73966eb40a41 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.920181 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.920843 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.921125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.921401 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.922087 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.924032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.946135 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1417/2814] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 08:59:51 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 08:59:51.950181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.960997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3020754-57bd-434e-a056-3dc060e6ab4d admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.966427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:51.967091 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:51.967410 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:51.967690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:51.968418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:51.970024 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:51.980636 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1398/2815] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:59:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:59:51.984793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:51.994783 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386d53c8-fb89-4e67-8ac7-e4aff4870179 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:51.999322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:52.000195 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:52.000476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:52.000753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] PATH_INFO: `/v3/projects/97ab253150904b80ad9291789a5b5aba/users/a0ad106c409e49e79edea066215ca075/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:52.002918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:52.017889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ab253150904b80ad9291789a5b5aba, user_id=a0ad106c409e49e79edea066215ca075, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:52.020545 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:52.034411 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1418/2816] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:59:51 2022] PUT /identity/v3/projects/97ab253150904b80ad9291789a5b5aba/users/a0ad106c409e49e79edea066215ca075/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:52.038991 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:52.049094 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af25f388-3391-4998-8342-9ce3878f91cd admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:52.061307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 08:59:52.107605 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5570-73d8-4645-8986-113fe1c4c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:52.108269 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5570-73d8-4645-8986-113fe1c4c924 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:52.108560 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5570-73d8-4645-8986-113fe1c4c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:52.108857 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5570-73d8-4645-8986-113fe1c4c924 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:52.109638 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5570-73d8-4645-8986-113fe1c4c924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:52.111460 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5570-73d8-4645-8986-113fe1c4c924 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:52.121803 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1399/2817] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 08:59:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 08:59:52.126050 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:52.136957 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb32c8a-1557-4daa-aabd-60b241932d4c admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:52.141521 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:52.142396 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:52.142679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:52.142953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] PATH_INFO: `/v3/projects/97ab253150904b80ad9291789a5b5aba/users/a0ad106c409e49e79edea066215ca075/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:52.144921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ab253150904b80ad9291789a5b5aba, user_id=a0ad106c409e49e79edea066215ca075, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:52.147582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:52.157729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1419/2818] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 08:59:52 2022] PUT /identity/v3/projects/97ab253150904b80ad9291789a5b5aba/users/a0ad106c409e49e79edea066215ca075/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 08:59:52.169362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:52.169685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:52.169982 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:52.198052 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] MFA Rules not processed for user `a0ad106c409e49e79edea066215ca075`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 08:59:52.199294 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:52.266439 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1400/2819] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 08:59:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 08:59:52.277686 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:52.279415 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 08:59:52.339159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885ea20e-ada9-4275-aad9-95b6d644e44f admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:52.342716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:52.343420 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:52.343709 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:52.343990 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:52.352962 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:52.354389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:52.363795 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1420/2820] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:52 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 08:59:52.434298 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 08:59:52.444416 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b8b95a-8322-4189-a979-a523d9685e69 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 08:59:52.448608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a510b6-96e3-4abe-8cce-ffdb44e88989 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 08:59:52.449240 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a510b6-96e3-4abe-8cce-ffdb44e88989 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 08:59:52.449520 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a510b6-96e3-4abe-8cce-ffdb44e88989 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 08:59:52.449792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a510b6-96e3-4abe-8cce-ffdb44e88989 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 08:59:52.505508 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a510b6-96e3-4abe-8cce-ffdb44e88989 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 08:59:52.506949 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a510b6-96e3-4abe-8cce-ffdb44e88989 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 08:59:52.516332 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1401/2821] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 08:59:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:02.351546 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:02.413650 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f4207-a610-4194-9499-f9b6beb308d1 service cinder] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:02.418316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:02.419193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:02.419510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:02.419793 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] PATH_INFO: `/v3/users/e9fd13fa406b4fa5bbf75d8af47e9c2e` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:02.420981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fd13fa406b4fa5bbf75d8af47e9c2e)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:02.422918 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:02.471573 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fd13fa406b4fa5bbf75d8af47e9c2e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:02.478450 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fd13fa406b4fa5bbf75d8af47e9c2e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:02.486164 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fd13fa406b4fa5bbf75d8af47e9c2e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:02.487034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fd13fa406b4fa5bbf75d8af47e9c2e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:02.487776 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fd13fa406b4fa5bbf75d8af47e9c2e'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:02.496629 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:02.511436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1421/2822] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:02 2022] DELETE /identity/v3/users/e9fd13fa406b4fa5bbf75d8af47e9c2e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:02.512263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=84 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:02.516129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:02.516657 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:02.517185 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:02.534629 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:02.535910 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:02.596166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1402/2823] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:02.606034 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:02.607526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:02.665154 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e26236-b2e4-4a4d-9a32-49fad3fddabc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:02.669326 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:02.669984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:02.670270 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:02.670547 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:02.681298 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:02.682731 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:02.692322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1422/2824] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:02.974924 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:02.976928 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:02.989140 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8b1330-0913-44ec-9aea-5622557c846f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:02.994342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:02.995242 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:02.995529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:02.995865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] PATH_INFO: `/v3/projects/4e180bbd7b9c4e81a09db5a5e7fdc1bf` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:02.997010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e180bbd7b9c4e81a09db5a5e7fdc1bf)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:02.999021 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:03.046249 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e180bbd7b9c4e81a09db5a5e7fdc1bf', 'request_id': 'req-16b471e3-a085-484e-8c8e-a4ea7469fc33'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.055506 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e180bbd7b9c4e81a09db5a5e7fdc1bf', 'request_id': 'req-16b471e3-a085-484e-8c8e-a4ea7469fc33'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.061860 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e180bbd7b9c4e81a09db5a5e7fdc1bf', 'request_id': 'req-16b471e3-a085-484e-8c8e-a4ea7469fc33'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.072505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] The token cache is being invalidate because project 4e180bbd7b9c4e81a09db5a5e7fdc1bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:03.072897 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.073446 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.074986 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1403/2825] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:02 2022] DELETE /identity/v3/projects/4e180bbd7b9c4e81a09db5a5e7fdc1bf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:03.079011 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:03.088755 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119aaff7-502b-408c-84a8-5429b4b15f1c service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:03.146090 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:03.146993 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.147309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.147593 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] PATH_INFO: `/v3/users/a0ad106c409e49e79edea066215ca075` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.148780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0ad106c409e49e79edea066215ca075)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:03.150687 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:03.206555 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0ad106c409e49e79edea066215ca075'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.213732 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0ad106c409e49e79edea066215ca075'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.221467 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0ad106c409e49e79edea066215ca075'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.222123 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0ad106c409e49e79edea066215ca075'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.222738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-16f83000-0bdc-49b8-a897-379fc8ae50f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0ad106c409e49e79edea066215ca075'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.240583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1423/2826] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:03 2022] DELETE /identity/v3/users/a0ad106c409e49e79edea066215ca075 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:03.506801 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:03.508322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:03.523385 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b471e3-a085-484e-8c8e-a4ea7469fc33 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:03.581154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:03.582050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.582369 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.582651 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] PATH_INFO: `/v3/projects/97ab253150904b80ad9291789a5b5aba` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.583883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ab253150904b80ad9291789a5b5aba)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:03.586499 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:03.616216 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:03.634978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ab253150904b80ad9291789a5b5aba', 'request_id': 'req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.641771 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ab253150904b80ad9291789a5b5aba', 'request_id': 'req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.648614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ab253150904b80ad9291789a5b5aba', 'request_id': 'req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.660308 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] The token cache is being invalidate because project 97ab253150904b80ad9291789a5b5aba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:03.660585 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.661223 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-0af6b92c-7ac7-43ce-bbc2-d5f8184570dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:03.662758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1404/2827] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:03 2022] DELETE /identity/v3/projects/97ab253150904b80ad9291789a5b5aba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:03.669113 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.669763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.669763 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.688340 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:03.689584 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:03.698379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:03.751140 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1424/2828] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:03.760487 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.760702 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.760975 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.781166 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:03.782381 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:03.790457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1405/2829] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:03.794936 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:03.805212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dacd7-2fd8-4f75-953e-57a478bf81f2 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:03.809764 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:03.810596 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.810729 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.811094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.811737 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:00:03.812048 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:03.813644 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:03.825754 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1425/2830] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 09:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:00:03.830330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:03.840313 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d298e-17c2-4ade-b0a9-8041553b1db5 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:03.844984 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:03.845625 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.845903 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.846173 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.846845 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:03.848756 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:03.873534 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1406/2831] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:03 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:03.877837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:03.888327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e455383-fb35-44f9-bc89-ed9401a04eb8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:03.946436 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:03.947114 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.947446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.947728 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.948429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:03.950367 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:03.970996 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1426/2832] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:03 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:03.975229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:03.985480 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a58027-89bb-4217-b306-fe6c01ceb882 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:03.990074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:03.990734 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:03.991019 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:03.991342 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:03.992073 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:03.993673 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.004815 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1407/2833] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:04.008622 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.018617 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b4c311-5adb-4c38-9d44-5f79089f08b8 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.023690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48092ed-18ee-4b20-991e-84f02dd67945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.024343 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48092ed-18ee-4b20-991e-84f02dd67945 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.024626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48092ed-18ee-4b20-991e-84f02dd67945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.024902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48092ed-18ee-4b20-991e-84f02dd67945 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.025621 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48092ed-18ee-4b20-991e-84f02dd67945 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.027192 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48092ed-18ee-4b20-991e-84f02dd67945 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.037749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1427/2834] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:04.041795 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.051965 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d51221-a786-405e-8567-5090df49f588 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.057064 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.057971 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.058253 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.058542 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] PATH_INFO: `/v3/projects/58dbfeee25e44a1fa4c3d8b139319ce4/users/3c19bcd679ed427f942adbb374afe86b/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.073518 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58dbfeee25e44a1fa4c3d8b139319ce4, user_id=3c19bcd679ed427f942adbb374afe86b, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.076243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.089078 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1408/2835] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:04 2022] PUT /identity/v3/projects/58dbfeee25e44a1fa4c3d8b139319ce4/users/3c19bcd679ed427f942adbb374afe86b/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:04.101036 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.101327 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.101607 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.129679 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] MFA Rules not processed for user `3c19bcd679ed427f942adbb374afe86b`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:04.130884 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:04.196472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1428/2836] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:04 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:04.201125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.263766 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eac3b3-4d32-42d0-8b0a-53cb49861ff0 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.268318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.269028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.269248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.269522 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.270212 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.272147 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.297812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1409/2837] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:04 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:00:04.302866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.320837 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d27b50-b58b-4108-bb49-a360222ed809 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.325490 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:04.379409 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.380118 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.380363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.380648 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.381331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.383951 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.406465 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1429/2838] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:04 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:04.410454 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.420796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6e9a8c-6ee5-4c3b-b4f4-7b7cd293176c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.425391 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.426123 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.426402 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.426676 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.427694 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.429792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.440495 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1410/2839] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:04.445228 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.455142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e89c3-f700-4836-b679-99617d45fad4 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.460363 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.461253 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.461533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.461810 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] PATH_INFO: `/v3/projects/1677e9286cb74101ab6b1490d53dc4e9/users/0fc420a386ad42debf2a97abfae11889/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.477341 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1677e9286cb74101ab6b1490d53dc4e9, user_id=0fc420a386ad42debf2a97abfae11889, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.480007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.493624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1430/2840] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:04 2022] PUT /identity/v3/projects/1677e9286cb74101ab6b1490d53dc4e9/users/0fc420a386ad42debf2a97abfae11889/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:04.497653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.507648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586d2165-7be7-4211-a676-bba742d94388 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.527055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:04.568172 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c87f6e-b55b-4c38-b434-a5095ebfa9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.568844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87f6e-b55b-4c38-b434-a5095ebfa9da admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.569127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87f6e-b55b-4c38-b434-a5095ebfa9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.569423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c87f6e-b55b-4c38-b434-a5095ebfa9da admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.570170 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c87f6e-b55b-4c38-b434-a5095ebfa9da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.571844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c87f6e-b55b-4c38-b434-a5095ebfa9da admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.581639 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1411/2841] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:04 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:04.585902 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.596264 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7489e2cf-6312-4517-914d-06459822856e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.601427 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.602352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.602634 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.602909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] PATH_INFO: `/v3/projects/1677e9286cb74101ab6b1490d53dc4e9/users/0fc420a386ad42debf2a97abfae11889/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.604876 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1677e9286cb74101ab6b1490d53dc4e9, user_id=0fc420a386ad42debf2a97abfae11889, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.607562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.618182 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1431/2842] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:04 2022] PUT /identity/v3/projects/1677e9286cb74101ab6b1490d53dc4e9/users/0fc420a386ad42debf2a97abfae11889/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:04.628496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.628791 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.629072 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.654822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] MFA Rules not processed for user `0fc420a386ad42debf2a97abfae11889`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:04.656161 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:04.722846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1412/2843] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:04 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:04.735108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:04.736921 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:04.794519 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dcd55-cdde-41ec-b4ff-29490b2d3aaf admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:04.798007 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db0e338-74fe-4e89-9989-bcbde1e24914 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:04.798645 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0e338-74fe-4e89-9989-bcbde1e24914 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:04.798926 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0e338-74fe-4e89-9989-bcbde1e24914 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:04.799201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db0e338-74fe-4e89-9989-bcbde1e24914 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:04.809196 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0e338-74fe-4e89-9989-bcbde1e24914 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:04.810612 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db0e338-74fe-4e89-9989-bcbde1e24914 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:04.820085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1432/2844] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:04 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:06.155376 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:06.222658 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf9d845-e4ed-40d7-afe9-8e932f75e731 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:06.227678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:06.228559 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:06.228844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:06.229125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] PATH_INFO: `/v3/users/3c19bcd679ed427f942adbb374afe86b` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:06.230311 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c19bcd679ed427f942adbb374afe86b)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:06.232276 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:06.278492 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c19bcd679ed427f942adbb374afe86b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.280160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:06.287150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c19bcd679ed427f942adbb374afe86b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.299529 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c19bcd679ed427f942adbb374afe86b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.306997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c19bcd679ed427f942adbb374afe86b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.307684 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c19bcd679ed427f942adbb374afe86b'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.313465 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1413/2845] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:06 2022] DELETE /identity/v3/users/3c19bcd679ed427f942adbb374afe86b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:06.317947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:06.318431 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:06.318579 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:06.336290 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:06.337312 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:06.397176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1433/2846] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:06.407538 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:06.408969 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:06.466160 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa7801-fe17-4c23-852a-4560b10c86c9 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:06.469812 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:06.470457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:06.470739 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:06.471016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:06.480802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:06.482248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:06.491794 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1414/2847] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:06.764626 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:06.766077 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:06.775092 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:06.780232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de69b2b-94e1-44b7-9acf-482ca78a4c70 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:06.786110 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:06.786973 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:06.787858 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:06.788151 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] PATH_INFO: `/v3/projects/58dbfeee25e44a1fa4c3d8b139319ce4` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:06.789587 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58dbfeee25e44a1fa4c3d8b139319ce4)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:06.791610 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:06.838697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58dbfeee25e44a1fa4c3d8b139319ce4', 'request_id': 'req-dcaa3770-c527-46b6-92c2-be8b1bfca842'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.844350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58dbfeee25e44a1fa4c3d8b139319ce4', 'request_id': 'req-dcaa3770-c527-46b6-92c2-be8b1bfca842'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.850538 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58dbfeee25e44a1fa4c3d8b139319ce4', 'request_id': 'req-dcaa3770-c527-46b6-92c2-be8b1bfca842'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.863403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] The token cache is being invalidate because project 58dbfeee25e44a1fa4c3d8b139319ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:06.863690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.864322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:06.865855 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1434/2848] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:06 2022] DELETE /identity/v3/projects/58dbfeee25e44a1fa4c3d8b139319ce4 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:00:06.869822 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:06.879564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfa7dac-8950-4965-9743-8c658a5c71af service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:06.943010 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:06.945039 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:06.945346 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:06.945630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] PATH_INFO: `/v3/users/0fc420a386ad42debf2a97abfae11889` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:06.947055 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc420a386ad42debf2a97abfae11889)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:06.949048 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:06.995827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc420a386ad42debf2a97abfae11889'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.003199 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc420a386ad42debf2a97abfae11889'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.014434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:07.015690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc420a386ad42debf2a97abfae11889'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.024900 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc420a386ad42debf2a97abfae11889'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.025609 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-90a78dc8-38d6-44a2-8256-c7732617d5b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc420a386ad42debf2a97abfae11889'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.031500 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1415/2849] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:06 2022] DELETE /identity/v3/users/0fc420a386ad42debf2a97abfae11889 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:07.312889 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.314350 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:07.328122 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa3770-c527-46b6-92c2-be8b1bfca842 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.388157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.388875 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.389156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.389438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] PATH_INFO: `/v3/projects/1677e9286cb74101ab6b1490d53dc4e9` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.390970 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1677e9286cb74101ab6b1490d53dc4e9)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.393540 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.441605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1677e9286cb74101ab6b1490d53dc4e9', 'request_id': 'req-87d2d3e4-f2ba-483f-b413-61dc094fed7a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.447533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1677e9286cb74101ab6b1490d53dc4e9', 'request_id': 'req-87d2d3e4-f2ba-483f-b413-61dc094fed7a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.453867 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1677e9286cb74101ab6b1490d53dc4e9', 'request_id': 'req-87d2d3e4-f2ba-483f-b413-61dc094fed7a'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.466241 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] The token cache is being invalidate because project 1677e9286cb74101ab6b1490d53dc4e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:07.466613 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.467279 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-87d2d3e4-f2ba-483f-b413-61dc094fed7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:07.468961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1435/2850] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:07 2022] DELETE /identity/v3/projects/1677e9286cb74101ab6b1490d53dc4e9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:07.473914 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.474419 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.474525 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.493056 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:07.494254 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:07.552003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1416/2851] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:07.560948 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.561331 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.561619 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.581351 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:07.582574 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:07.590818 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1436/2852] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 24 09:00:07.595143 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.605663 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f21264-b8d7-4dcd-b9c0-fa3ab39f16a4 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.610630 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.611347 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.611634 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.611911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.612648 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:00:07.612954 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.614573 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.627277 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1417/2853] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 09:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:00:07.632097 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.642207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f04436d-975a-4898-9a9b-885245b0c5fa None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.647533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.648210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.648491 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.648765 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.649476 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.651421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.660347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:07.687882 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1437/2854] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:07 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:07.692179 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.702588 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2f114-0b01-40a0-9b5a-4570d622affc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.762318 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.762999 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.763374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.763665 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.764388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.766353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.786400 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1418/2855] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:07 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:07.791085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.801662 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09947992-fc61-406a-80ca-31926362d476 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.807425 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.808081 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.808389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.808684 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.809404 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.810981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.820953 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1438/2856] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:07.824913 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.835236 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dbfa5-3bbe-4fb9-9a2e-d561e471ac8e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.840106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933e9abe-d3a2-4dd5-8951-882b819eab02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.840758 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e9abe-d3a2-4dd5-8951-882b819eab02 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.841040 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e9abe-d3a2-4dd5-8951-882b819eab02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.841317 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933e9abe-d3a2-4dd5-8951-882b819eab02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.842071 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e9abe-d3a2-4dd5-8951-882b819eab02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.843685 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933e9abe-d3a2-4dd5-8951-882b819eab02 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.854395 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1419/2857] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:07.859480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:07.871106 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38db21b-401c-41cc-a483-a7af86a3f5cc admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:07.875866 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:07.876749 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.877035 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.877362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] PATH_INFO: `/v3/projects/45ea4706dcdd47339d2624b4c3223ce5/users/dc870646e0874a2881aa32ae31d1ab02/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.893501 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] RBAC: Authorizing `identity:create_grant(project_id=45ea4706dcdd47339d2624b4c3223ce5, user_id=dc870646e0874a2881aa32ae31d1ab02, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:07.896246 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:07.910974 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1439/2858] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:07 2022] PUT /identity/v3/projects/45ea4706dcdd47339d2624b4c3223ce5/users/dc870646e0874a2881aa32ae31d1ab02/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:07.923270 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:07.923567 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:07.923844 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:07.952450 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] MFA Rules not processed for user `dc870646e0874a2881aa32ae31d1ab02`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:07.953594 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:07.996938 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:08.020524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1420/2859] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:08.025950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.096089 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ef9013-748b-40d3-bb75-308d3bfa839e admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.101493 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.102156 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.102438 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.102714 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.103446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.105381 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.136725 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1440/2860] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:08 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 24 09:00:08.141263 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.152350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83b8176-8b9b-41f1-a917-d87599b452ca None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.238065 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.238792 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.239183 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.239551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.240299 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.242989 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.277728 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1421/2861] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:08 2022] POST /identity/v3/users => generated 295 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:08.281923 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.293525 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82c37b2-66ab-4adb-8cf7-7a084fe6c7ff admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.298352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.299032 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.299362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.299642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.300418 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.302014 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.314605 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1441/2862] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:08.319483 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.330505 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21334e-12ea-47eb-b053-3ad4a00f804e admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.337007 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.337909 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.338233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.338512 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] PATH_INFO: `/v3/projects/a0b6caf640554a4a92f3341314bedf2a/users/e7fbaac93d314160974b9d0ef2fb16a5/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.356375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b6caf640554a4a92f3341314bedf2a, user_id=e7fbaac93d314160974b9d0ef2fb16a5, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.359241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.375933 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1422/2863] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:08 2022] PUT /identity/v3/projects/a0b6caf640554a4a92f3341314bedf2a/users/e7fbaac93d314160974b9d0ef2fb16a5/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:08.380185 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.391472 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ca7bc-4c8b-44f7-ac02-a12bfc33af0b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.487762 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b566001-50b9-4f52-ac33-447675a6d111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.488947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b566001-50b9-4f52-ac33-447675a6d111 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.489353 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b566001-50b9-4f52-ac33-447675a6d111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.489841 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b566001-50b9-4f52-ac33-447675a6d111 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.491190 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b566001-50b9-4f52-ac33-447675a6d111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.494819 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b566001-50b9-4f52-ac33-447675a6d111 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.514591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1442/2864] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:08 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:08.519700 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.538796 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2f9676-821f-403d-ad73-aa13d2950850 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.547181 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.548610 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.548953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.549238 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] PATH_INFO: `/v3/projects/a0b6caf640554a4a92f3341314bedf2a/users/e7fbaac93d314160974b9d0ef2fb16a5/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.551475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0b6caf640554a4a92f3341314bedf2a, user_id=e7fbaac93d314160974b9d0ef2fb16a5, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.555326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.567678 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1423/2865] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:08 2022] PUT /identity/v3/projects/a0b6caf640554a4a92f3341314bedf2a/users/e7fbaac93d314160974b9d0ef2fb16a5/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:08.579984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.580526 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.580903 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.617835 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] MFA Rules not processed for user `e7fbaac93d314160974b9d0ef2fb16a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:08.619852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:08.646443 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:08.715286 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1443/2866] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:08.727540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:08.729322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:08.792074 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ebbf54-f8fe-4846-bbdf-ac9c417a535c admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:08.796597 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:08.797330 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:08.797681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:08.798050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:08.857619 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:08.869575 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:08.871481 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:08.882687 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1424/2867] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:14.665658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:14.667125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:14.679562 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51bde5a-0372-4834-ba06-8d9c23c96852 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:14.683417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50bea0-c682-4b3a-ada7-60db1bba1655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:14.684403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50bea0-c682-4b3a-ada7-60db1bba1655 service cinder] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:14.684403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50bea0-c682-4b3a-ada7-60db1bba1655 service cinder] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:14.684630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50bea0-c682-4b3a-ada7-60db1bba1655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:14.694027 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50bea0-c682-4b3a-ada7-60db1bba1655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:14.695535 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50bea0-c682-4b3a-ada7-60db1bba1655 service cinder] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:14.706456 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1444/2868] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 24 09:00:18.918602 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=5 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:24.701051 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=29 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:42.847413 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:42.913501 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb6e0d-3838-4ca6-994f-4e6b90cf4d81 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:42.918061 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:42.918963 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:42.919274 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:42.919556 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] PATH_INFO: `/v3/users/dc870646e0874a2881aa32ae31d1ab02` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:42.921958 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc870646e0874a2881aa32ae31d1ab02)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:42.923953 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:42.974365 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc870646e0874a2881aa32ae31d1ab02'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:42.983720 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc870646e0874a2881aa32ae31d1ab02'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:42.999063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc870646e0874a2881aa32ae31d1ab02'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.006062 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc870646e0874a2881aa32ae31d1ab02'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.006922 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc870646e0874a2881aa32ae31d1ab02'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.013373 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1425/2869] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:42 2022] DELETE /identity/v3/users/dc870646e0874a2881aa32ae31d1ab02 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:43.018144 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:43.018726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:43.018726 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:43.036820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:43.038692 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:43.104585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1445/2870] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:43.114112 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:43.115586 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:43.186690 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a023fe-2fb6-4750-97e7-107230462799 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:43.190353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:43.191003 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:43.191319 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:43.191600 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:43.201211 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:43.202666 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:43.212865 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1426/2871] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:43.485463 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:43.486940 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:43.499635 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e6f5da-67b0-41d2-9597-ef6260ee5f36 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:43.507041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:43.508421 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:43.508856 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:43.509311 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] PATH_INFO: `/v3/projects/45ea4706dcdd47339d2624b4c3223ce5` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:43.511094 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45ea4706dcdd47339d2624b4c3223ce5)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:43.514292 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:43.577159 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45ea4706dcdd47339d2624b4c3223ce5', 'request_id': 'req-beac044b-2d16-4b4d-b030-24bcc0a03160'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.582690 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45ea4706dcdd47339d2624b4c3223ce5', 'request_id': 'req-beac044b-2d16-4b4d-b030-24bcc0a03160'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.589524 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45ea4706dcdd47339d2624b4c3223ce5', 'request_id': 'req-beac044b-2d16-4b4d-b030-24bcc0a03160'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.603274 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] The token cache is being invalidate because project 45ea4706dcdd47339d2624b4c3223ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:43.603565 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.604207 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.605790 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1446/2872] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:43 2022] DELETE /identity/v3/projects/45ea4706dcdd47339d2624b4c3223ce5 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:43.609850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:43.619622 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07013632-8f8a-40f1-92f3-c19538f44d5a service neutron] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:43.638620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:43.680799 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:43.681683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:43.681966 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:43.682241 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] PATH_INFO: `/v3/users/e7fbaac93d314160974b9d0ef2fb16a5` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:43.683470 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7fbaac93d314160974b9d0ef2fb16a5)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:43.685530 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:43.733375 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7fbaac93d314160974b9d0ef2fb16a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.740955 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7fbaac93d314160974b9d0ef2fb16a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.752748 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7fbaac93d314160974b9d0ef2fb16a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.759972 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7fbaac93d314160974b9d0ef2fb16a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.760650 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-19ca5256-1eef-43c0-8826-7bedca92562c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7fbaac93d314160974b9d0ef2fb16a5'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:43.769016 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1427/2873] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:43 2022] DELETE /identity/v3/users/e7fbaac93d314160974b9d0ef2fb16a5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:44.045904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.047681 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:44.067485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac044b-2d16-4b4d-b030-24bcc0a03160 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.124750 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.125534 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.125820 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.126098 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] PATH_INFO: `/v3/projects/a0b6caf640554a4a92f3341314bedf2a` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.127310 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0b6caf640554a4a92f3341314bedf2a)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.129212 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.160695 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:44.180148 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0b6caf640554a4a92f3341314bedf2a', 'request_id': 'req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:44.185697 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0b6caf640554a4a92f3341314bedf2a', 'request_id': 'req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:44.192108 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0b6caf640554a4a92f3341314bedf2a', 'request_id': 'req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:44.213131 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] The token cache is being invalidate because project a0b6caf640554a4a92f3341314bedf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80406) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:44.213422 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:44.214064 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-0dff42f6-e579-4b1d-abf3-d6a40fd3c9d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:44.215664 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1447/2874] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:44 2022] DELETE /identity/v3/projects/a0b6caf640554a4a92f3341314bedf2a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:44.220686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.221192 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.221302 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.238967 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:44.240229 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:44.299228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1428/2875] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:44.359854 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.360193 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.360478 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.378812 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:44.379984 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:44.388460 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1448/2876] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:44.392742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.403243 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3f215f-5b1c-4692-9a3c-f33ade1c5325 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.407827 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.408486 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.408774 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.409050 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] PATH_INFO: `/v3/domains` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.409776 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80407) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:00:44.410083 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.411725 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.423524 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1429/2877] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 09:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:00:44.427949 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.437385 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a29ac6a-d4a6-4eb9-91ac-ee951e16cd0a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.442101 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.442759 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.443047 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.443361 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.444550 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.446451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.472278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1449/2878] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:44.476475 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.486880 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d1ff5-1bff-4520-9e4b-11be50eb6ebe admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.513321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:44.546352 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.547029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.547350 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.547742 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.548441 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.550379 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.575443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1430/2879] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:44.576802 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.588074 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97a97e-183a-475a-bd18-74a770402ffa admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.593446 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.594147 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.594434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.594712 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.595480 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.597088 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.609389 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1450/2880] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:44.613686 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.624509 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e477cc-1be9-42ff-a971-aea26747fd3a admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.629322 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98488640-7247-419d-8900-753e8dfe3101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.629952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488640-7247-419d-8900-753e8dfe3101 admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.630232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488640-7247-419d-8900-753e8dfe3101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.630504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98488640-7247-419d-8900-753e8dfe3101 admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.631337 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488640-7247-419d-8900-753e8dfe3101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.633540 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98488640-7247-419d-8900-753e8dfe3101 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.644493 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1431/2881] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:44.649976 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.660201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ae9e5-bfc6-4794-ab5c-1a3facda3ac5 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.665604 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.666500 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] REQUEST_METHOD: `PUT` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.666786 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.667063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] PATH_INFO: `/v3/projects/33389dd706cd41999d13642c2b4d2cc3/users/aa10bac78eec48d49de6481a8318a313/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.682137 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] RBAC: Authorizing `identity:create_grant(project_id=33389dd706cd41999d13642c2b4d2cc3, user_id=aa10bac78eec48d49de6481a8318a313, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.684880 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.698272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1451/2882] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:44 2022] PUT /identity/v3/projects/33389dd706cd41999d13642c2b4d2cc3/users/aa10bac78eec48d49de6481a8318a313/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:44.708321 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.708614 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.708899 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.736232 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] MFA Rules not processed for user `aa10bac78eec48d49de6481a8318a313`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:44.737451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:44.804228 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1432/2883] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:44.808805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.871157 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98601c4-23fa-4156-8d29-ea54903475be admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.875907 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.876572 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.876860 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.877142 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.877849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:44.879852 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:44.907124 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1452/2884] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:44.913504 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:44.925703 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad8536-8f52-4f2e-aa4a-0cf0e4ee39aa None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:44.996154 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:44.996838 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:44.997220 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:44.997577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] PATH_INFO: `/v3/users` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:44.998272 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.000248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.026129 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1433/2885] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:44 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:45.027507 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.038668 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f7b194-1297-4074-811f-e13dda567a09 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.043741 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.044434 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.044716 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.045005 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.045718 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.047382 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.059075 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1453/2886] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:45.063423 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.074037 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3f5e04-1ec3-4038-baa0-b7d14d578478 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.078800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.079737 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.080018 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.080297 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] PATH_INFO: `/v3/projects/c413321ab8064f3f8111f49521430823/users/bc193814cfda488b9dd50ed1065eba68/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.096180 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c413321ab8064f3f8111f49521430823, user_id=bc193814cfda488b9dd50ed1065eba68, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.098876 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.113471 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1434/2887] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:45 2022] PUT /identity/v3/projects/c413321ab8064f3f8111f49521430823/users/bc193814cfda488b9dd50ed1065eba68/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:45.118886 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.128853 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0954b9e3-f39b-4035-8feb-65e994f717ac admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.168235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:45.216552 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e1646-503c-4323-93bc-c482a7f65035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.217225 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e1646-503c-4323-93bc-c482a7f65035 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.217512 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e1646-503c-4323-93bc-c482a7f65035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.217782 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e1646-503c-4323-93bc-c482a7f65035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.218499 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e1646-503c-4323-93bc-c482a7f65035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.220532 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0e1646-503c-4323-93bc-c482a7f65035 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.230938 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1454/2888] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:45 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 24 09:00:45.236111 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.242139 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:45.258850 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8c075-f002-42f2-8a31-6e0184294c41 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.268732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.269935 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.271357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.271357 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] PATH_INFO: `/v3/projects/c413321ab8064f3f8111f49521430823/users/bc193814cfda488b9dd50ed1065eba68/roles/b382dafaa79d4ff1afb67e7c0d7097a8` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.274201 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c413321ab8064f3f8111f49521430823, user_id=bc193814cfda488b9dd50ed1065eba68, role_id=b382dafaa79d4ff1afb67e7c0d7097a8)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.277067 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.292135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1435/2889] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:45 2022] PUT /identity/v3/projects/c413321ab8064f3f8111f49521430823/users/bc193814cfda488b9dd50ed1065eba68/roles/b382dafaa79d4ff1afb67e7c0d7097a8 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:00:45.309380 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.309868 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.310335 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.341768 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] MFA Rules not processed for user `bc193814cfda488b9dd50ed1065eba68`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:45.342980 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:45.417892 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1455/2890] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:45 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:45.431128 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.432883 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:45.517106 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8ae37-6bf9-42cb-8752-7bdd8dc6bc29 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.520800 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.521448 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.521732 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.522038 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.532554 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.534020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.543952 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1436/2891] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:45 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:45.846582 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.857963 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef7081-4621-44b9-b247-2efb46210c1a None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.863062 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc193814cfda488b9dd50ed1065eba68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c413321ab8064f3f8111f49521430823', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.863784 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.864096 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.864384 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.865125 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:00:45.865451 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.867242 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.879205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1456/2892] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 09:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:00:45.883911 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.893372 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4ff827-e2cd-4a64-9d49-7699bb0f0e37 service cinder] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.898476 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc193814cfda488b9dd50ed1065eba68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c413321ab8064f3f8111f49521430823', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:45.899127 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:45.899451 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:45.899743 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:45.900443 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:45.902358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:45.928571 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1437/2893] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:45 2022] POST /identity/v3/projects => generated 355 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:45.932806 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:45.942904 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72878d21-a3d0-4fe6-9806-6db1506831e5 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:45.999910 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc193814cfda488b9dd50ed1065eba68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c413321ab8064f3f8111f49521430823', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:46.000591 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:46.000878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:46.001165 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:46.001897 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:00:46.002211 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:46.003849 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:46.016218 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1457/2894] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 09:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 24 09:00:46.053472 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:46.064326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44d030-029d-4395-96a0-58ec9de46bff tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:46.069577 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc193814cfda488b9dd50ed1065eba68', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c413321ab8064f3f8111f49521430823', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:46.070464 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:46.070749 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:46.071028 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] PATH_INFO: `/v3/projects/3925c0d1cf654533a6c2f05363a9e032` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:46.072248 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3925c0d1cf654533a6c2f05363a9e032)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:46.074200 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:46.121832 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3925c0d1cf654533a6c2f05363a9e032', 'request_id': 'req-d71779d7-a5b0-4e7e-a208-0f18efc01c23'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.129310 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3925c0d1cf654533a6c2f05363a9e032', 'request_id': 'req-d71779d7-a5b0-4e7e-a208-0f18efc01c23'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.136789 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3925c0d1cf654533a6c2f05363a9e032', 'request_id': 'req-d71779d7-a5b0-4e7e-a208-0f18efc01c23'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.147104 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] The token cache is being invalidate because project 3925c0d1cf654533a6c2f05363a9e032 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:46.147437 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.148097 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-d71779d7-a5b0-4e7e-a208-0f18efc01c23 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.149677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1438/2895] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:46 2022] DELETE /identity/v3/projects/3925c0d1cf654533a6c2f05363a9e032 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:46.341176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:46.342919 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:46.374303 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=100 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:46.408176 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66af25c5-573a-4fa4-a74a-415c3808fd39 tempest-VolumeQuotaClassesTest-72306645 tempest-VolumeQuotaClassesTest-72306645-project-admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:46.464433 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:46.465321 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:46.465609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:46.465890 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] PATH_INFO: `/v3/users/aa10bac78eec48d49de6481a8318a313` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:46.467126 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa10bac78eec48d49de6481a8318a313)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:46.469068 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:46.515950 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa10bac78eec48d49de6481a8318a313'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.523505 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa10bac78eec48d49de6481a8318a313'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.531585 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa10bac78eec48d49de6481a8318a313'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.532205 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa10bac78eec48d49de6481a8318a313'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.532805 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa10bac78eec48d49de6481a8318a313'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:46.550136 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1458/2896] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:46 2022] DELETE /identity/v3/users/aa10bac78eec48d49de6481a8318a313 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:46.554677 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:46.555187 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:46.555326 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:46.572853 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:46.573996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:46.604675 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:46.637457 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1439/2897] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:46.647887 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:46.649296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:46.705807 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebc5cbb-46ca-46d5-80bb-db341c1f9664 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:46.709965 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:46.710609 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:46.710891 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:46.711168 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:46.718316 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:46.720878 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:46.722671 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:46.734258 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1459/2898] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:46.999964 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:47.001466 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:47.013566 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d9aca-85f9-44f2-a23e-0d05c9eda1b1 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:47.018198 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:47.019063 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.019377 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.019653 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] PATH_INFO: `/v3/projects/33389dd706cd41999d13642c2b4d2cc3` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.020846 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=33389dd706cd41999d13642c2b4d2cc3)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:47.023291 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:47.070307 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33389dd706cd41999d13642c2b4d2cc3', 'request_id': 'req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.077353 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33389dd706cd41999d13642c2b4d2cc3', 'request_id': 'req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.083890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33389dd706cd41999d13642c2b4d2cc3', 'request_id': 'req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.095125 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] The token cache is being invalidate because project 33389dd706cd41999d13642c2b4d2cc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:47.095445 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.096101 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.097681 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1440/2899] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:46 2022] DELETE /identity/v3/projects/33389dd706cd41999d13642c2b4d2cc3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:47.102308 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:47.112235 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d24812-c865-44e4-93bd-fa7dda9aa465 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:47.169210 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:47.170093 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.170379 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.170658 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] PATH_INFO: `/v3/users/bc193814cfda488b9dd50ed1065eba68` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.171851 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc193814cfda488b9dd50ed1065eba68)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:47.173778 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:47.220665 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc193814cfda488b9dd50ed1065eba68'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.228366 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc193814cfda488b9dd50ed1065eba68'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.235968 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc193814cfda488b9dd50ed1065eba68'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.236667 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc193814cfda488b9dd50ed1065eba68'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.237306 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-7521f06c-5476-46c9-82d6-cfcd32ed72d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc193814cfda488b9dd50ed1065eba68'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.254698 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1460/2900] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:47 2022] DELETE /identity/v3/users/bc193814cfda488b9dd50ed1065eba68 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:47.531692 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:47.533306 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:47.547693 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff81a9-781b-4929-a6aa-c4c5a1ed1afc admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:47.607053 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:47.607868 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.608155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.608434 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] PATH_INFO: `/v3/projects/c413321ab8064f3f8111f49521430823` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.609644 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c413321ab8064f3f8111f49521430823)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:47.611607 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:47.657150 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c413321ab8064f3f8111f49521430823', 'request_id': 'req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.664044 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c413321ab8064f3f8111f49521430823', 'request_id': 'req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.665996 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:47.670596 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c413321ab8064f3f8111f49521430823', 'request_id': 'req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.682603 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] The token cache is being invalidate because project c413321ab8064f3f8111f49521430823 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:47.682893 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.683620 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-98ec826b-5ec2-4eb7-8b12-d1bf3dbe8ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:47.685358 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1441/2901] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:47 2022] DELETE /identity/v3/projects/c413321ab8064f3f8111f49521430823 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 24 09:00:47.690978 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.691533 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.691642 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.711640 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:47.712909 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:47.808632 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1461/2902] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:47.820397 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.820877 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.821367 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.850401 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:47.852234 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:47.864973 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1442/2903] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:47.871717 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:47.886838 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a834343-4d6d-4306-a7f7-35fd8b30d899 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:47.892360 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:47.893025 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.893307 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.893583 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] PATH_INFO: `/v3/domains` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.894309 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80406) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 24 09:00:47.894615 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:47.896272 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:47.910947 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1462/2904] 10.222.0.43 () {58 vars in 1152 bytes} [Fri Jun 24 09:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 24 09:00:47.915944 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:47.927817 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20937bf3-54b8-40a4-bf06-77b2f33c5167 None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:47.934138 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:47.934849 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:47.935132 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:47.935447 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:47.936153 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:47.938081 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:47.965020 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1443/2905] 10.222.0.43 () {60 vars in 1152 bytes} [Fri Jun 24 09:00:47 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:47.969639 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:47.975085 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:47.982278 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f36a5-ecf0-4c67-85b3-85247ddfb40d admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:48.042041 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:48.042711 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:48.042994 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:48.043296 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:48.043998 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:48.046079 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:48.067575 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1463/2906] 10.222.0.43 () {60 vars in 1143 bytes} [Fri Jun 24 09:00:47 2022] POST /identity/v3/users => generated 300 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 24 09:00:48.072233 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:48.083279 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e336886f-c71e-43ed-bbe8-a339649a26dd admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:48.088921 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:48.089608 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:48.089912 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:48.090186 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:48.090950 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:48.092611 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:48.105612 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1444/2907] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:48.109753 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:48.120347 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd13d7-9ecc-4178-af9b-e47bbe655fe9 admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:48.125362 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07638b18-e829-43d1-a0cd-e855f1863ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:48.126063 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638b18-e829-43d1-a0cd-e855f1863ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:48.126352 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638b18-e829-43d1-a0cd-e855f1863ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:48.126624 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07638b18-e829-43d1-a0cd-e855f1863ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:48.127403 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07638b18-e829-43d1-a0cd-e855f1863ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:48.128992 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07638b18-e829-43d1-a0cd-e855f1863ec9 admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:48.145954 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1464/2908] 10.222.0.43 () {58 vars in 1121 bytes} [Fri Jun 24 09:00:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 24 09:00:48.150696 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:48.161589 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906ce01-72e2-415b-bad1-af599d9a9f1f admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:48.167156 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:48.168135 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:48.168418 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:48.168683 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] PATH_INFO: `/v3/projects/1ab48a29337043fdac5ea37c32b8d3f2/users/9e5c331ecf064503a70cc40edc773e2f/roles/f64a7b4bd6b0489997f2c0dda8eed424` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:48.186246 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ab48a29337043fdac5ea37c32b8d3f2, user_id=9e5c331ecf064503a70cc40edc773e2f, role_id=f64a7b4bd6b0489997f2c0dda8eed424)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:48.188994 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:48.205155 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1445/2909] 10.222.0.43 () {60 vars in 1482 bytes} [Fri Jun 24 09:00:48 2022] PUT /identity/v3/projects/1ab48a29337043fdac5ea37c32b8d3f2/users/9e5c331ecf064503a70cc40edc773e2f/roles/f64a7b4bd6b0489997f2c0dda8eed424 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:48.225388 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] REQUEST_METHOD: `POST` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:48.225678 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:48.225961 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:48.255683 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.auth.core [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] MFA Rules not processed for user `9e5c331ecf064503a70cc40edc773e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80406) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:48.257417 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:48.325495 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1465/2910] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:48.338171 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:48.340029 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:48.400988 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec17c-a6ee-4427-b875-5635192cf005 admin admin] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:48.404564 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13e2ce7-4781-4779-a327-a62a314fca6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f3229f7fdd344dda97a47709c93ce2e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:48.405273 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2ce7-4781-4779-a327-a62a314fca6c service cinder] REQUEST_METHOD: `GET` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:48.405558 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2ce7-4781-4779-a327-a62a314fca6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:48.405831 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13e2ce7-4781-4779-a327-a62a314fca6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:48.414997 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e2ce7-4781-4779-a327-a62a314fca6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:48.416496 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13e2ce7-4781-4779-a327-a62a314fca6c service cinder] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:48.426221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1446/2911] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:56.505738 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:56.578743 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d8dbb4-6e80-42e0-9183-05bd2b36df84 None None] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:56.583322 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:56.584208 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:56.584494 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:56.584774 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] PATH_INFO: `/v3/users/9e5c331ecf064503a70cc40edc773e2f` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:56.585964 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5c331ecf064503a70cc40edc773e2f)` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:56.587941 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:56.601332 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:56.640981 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5c331ecf064503a70cc40edc773e2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:56.649066 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5c331ecf064503a70cc40edc773e2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:56.657870 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5c331ecf064503a70cc40edc773e2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:56.658504 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5c331ecf064503a70cc40edc773e2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:56.659201 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.notifications [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5c331ecf064503a70cc40edc773e2f'} {{(pid=80406) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:56.679232 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1466/2912] 10.222.0.43 () {58 vars in 1223 bytes} [Fri Jun 24 09:00:56 2022] DELETE /identity/v3/users/9e5c331ecf064503a70cc40edc773e2f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:00:56.683802 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] REQUEST_METHOD: `POST` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:56.684300 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:56.684427 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:56.701640 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.auth.core [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] MFA Rules not processed for user `ca1ff62a97334431beecc4658e2e8e61`. Rule list: `[]` (Enabled: `True`). {{(pid=80407) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 24 09:00:56.702858 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:56.772635 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1447/2913] 10.222.0.43 () {58 vars in 957 bytes} [Fri Jun 24 09:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 24 09:00:56.783509 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:56.785485 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:56.868187 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6a7b91-1f4c-47d3-908a-37d4c19e673b admin admin] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:56.872175 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '95d010330c9c44ddb09dbd8c501f3087', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:56.872982 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:56.873271 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:56.873580 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:56.884630 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:56.886091 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:56.896497 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1467/2914] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:00:57.186890 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] Authenticating user token {{(pid=80407) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:00:57.188362 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.fernet_utils [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80407) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:00:57.200978 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab7c1c-5a6c-4a62-8c9e-42e4ab82b25f None None] Validating token access rules against request {{(pid=80407) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:00:57.206221 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ca1ff62a97334431beecc4658e2e8e61', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffa0e0eb99bd42cebd2bc5b9d635fa5d', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80407) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:00:57.207093 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:00:57.207411 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] SCRIPT_NAME: `/identity` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:00:57.207691 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] PATH_INFO: `/v3/projects/1ab48a29337043fdac5ea37c32b8d3f2` {{(pid=80407) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:00:57.208894 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ab48a29337043fdac5ea37c32b8d3f2)` {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:00:57.211551 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] RBAC: Authorization granted {{(pid=80407) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:00:57.236388 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:00:57.265257 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ab48a29337043fdac5ea37c32b8d3f2', 'request_id': 'req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:57.273438 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ab48a29337043fdac5ea37c32b8d3f2', 'request_id': 'req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:57.281374 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ab48a29337043fdac5ea37c32b8d3f2', 'request_id': 'req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a'} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:57.292706 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] The token cache is being invalidate because project 1ab48a29337043fdac5ea37c32b8d3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80407) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 24 09:00:57.293043 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:57.294523 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG keystone.notifications [None req-ede6d1a7-4b7c-4e1a-9f5e-9e984c97040a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80407) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 24 09:00:57.296066 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: [pid: 80407|app: 0|req: 1448/2915] 10.222.0.43 () {58 vars in 1232 bytes} [Fri Jun 24 09:00:57 2022] DELETE /identity/v3/projects/1ab48a29337043fdac5ea37c32b8d3f2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 24 09:01:06.891694 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=48,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:01:07.289245 dv-cinder-845178-9-iscsi devstack@keystone.service[80407]: DEBUG dbcounter [-] [80407] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80407) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 24 09:04:54.184510 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] Authenticating user token {{(pid=80406) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 24 09:04:54.186086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:54.282554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b824eb-0df6-4a21-bb70-696c79b7a4e4 service neutron] Validating token access rules against request {{(pid=80406) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 24 09:04:54.353429 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a39a333f4c42420cb46189d32aafc6fb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '95ac8a1e51604baa804e78a0d3b026ce', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80406) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 24 09:04:54.354086 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] REQUEST_METHOD: `GET` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 24 09:04:54.354372 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] SCRIPT_NAME: `/identity` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 24 09:04:54.354646 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80406) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 24 09:04:54.356554 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.fernet_utils [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80406) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 24 09:04:54.368155 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 24 09:04:54.369733 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a28b63a-3cfa-485d-a6f6-1d4b2a73a83b service placement] RBAC: Authorization granted {{(pid=80406) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 24 09:04:54.379780 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: [pid: 80406|app: 0|req: 1468/2916] 10.222.0.43 () {60 vars in 1365 bytes} [Fri Jun 24 09:04:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 24 09:05:04.375468 dv-cinder-845178-9-iscsi devstack@keystone.service[80406]: DEBUG dbcounter [-] [80406] Writing DB stats keystone:SELECT=67 {{(pid=80406) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}